Replication and Meta-Analysis in Parapsychology

📌 Appears in:

⚡ Ganzfeld Telepathy ⚡ Meta Debate 📖 Steelman Psi

Plain English Summary

This is a big deal: a statistician from UC Davis got a paper about psychic research published in one of the most prestigious statistics journals around, complete with seven expert commentaries. Jessica Utts walked through a surprisingly common scientific blunder: when a replication study finds nothing significant, researchers often wrongly declare the original effect was fake, when really the new study just lacked enough statistical power (the ability to detect small effects). She then laid out meta-analyses (studies that pool many experiments together) across four types of psychic phenomena, and the numbers are genuinely striking. In telepathy experiments using the ganzfeld method (a sensory-deprivation setup), participants hit the right target 34.4% of the time versus the expected 25% by chance alone, with odds against luck of about 1 in 20,000. That effect size is actually triple the effect of aspirin in preventing heart attacks. The expert commentators were split but intrigued: two statisticians calculated the evidence at 100-to-200-to-1 in favor of something real happening. Utts concluded that a genuine anomaly exists that demands explanation.

Actual Paper Abstract

Parapsychology, the laboratory study of psychic phenomena, has had its history interwoven with that of statistics. Many of the controversies in parapsychology have focused on statistical issues, and statistical models have played an integral role in the experimental work. Recently, parapsychologists have been using meta-analysis as a tool for synthesizing large bodies of work. This paper presents an overview of the use of statistics in parapsychology and offers a summary of the meta-analyses that have been conducted. It begins with some anecdotal information about the involvement of statistics and statisticians with the early history of parapsychology. Next, it is argued that most nonstatisticians do not appreciate the connection between power and "successful" replication of experimental effects. Returning to parapsychology, a particular experimental regime is examined by summarizing an extended debate over the interpretation of the results. A new set of experiments designed to resolve the debate is then reviewed. Finally, meta-analyses from several areas of parapsychology are summarized. It is concluded that the overall evidence indicates that there is an anomalous effect in need of an explanation.

Research Notes

Landmark paper by UC Davis statistician in a premier statistics journal — one of very few psi papers published in a top-tier mainstream outlet. The seven invited commentaries are uniquely valuable: Diaconis demands qualified observers, Hyman concedes intriguing results, Bayarri & Berger calculate Bayes factor 100-200:1, Mosteller discusses null calibration. Essential for the meta-debate controversy and the ganzfeld telepathy debate.

Invited review in Statistical Science (Vol. 6, No. 4, pp. 363-403) with seven formal commentaries by Bayarri & Berger, Dawson, Diaconis, Greenhouse, Hyman, Morris, and Mosteller. Demonstrates through worked examples that misunderstandings of statistical power cause scientists to misinterpret nonsignificant replications as failures. Synthesizes meta-analyses across four psi domains: ganzfeld telepathy (autoganzfeld: 122/355 hits = 34.4% vs. 25% chance, p=0.00005, h=0.20), forced-choice precognition (309 studies, z=11.41), RNG/micro-PK (832 studies, z=4.1), and dice-PK (148 studies, z=18.2). Ganzfeld effect size triple that of aspirin on heart attacks. Concludes an anomalous effect exists requiring explanation.