Correcting the Past: Failures to Replicate Psi
🛡️ Critical replication ↗📌 Appears in:
Plain English Summary
Can people somehow study for a test *after* taking it and still boost their scores? That's essentially what Daryl Bem claimed in his famous 2011 experiments on precognition. A team of four researchers decided to put this to the ultimate test, running seven tightly controlled experiments with over 3,200 participants — no peeking at results early, no wiggling the methods. The verdict? Six out of seven experiments found absolutely nothing. The combined effect was essentially zero. A broader look at all 19 known replication attempts told the same story. Here's the real kicker, though: the only thing that predicted positive results was whether Bem himself ran the experiment. When he did, effects appeared; when anyone else tried, they vanished. This became a landmark study in the replication crisis — raising hard questions about whether subtle researcher choices, rather than psychic powers, drove the original findings.
Actual Paper Abstract
Across 7 experiments (N 3,289), we replicate the procedure of Experiments 8 and 9 from Bem (2011), which had originally demonstrated retroactive facilitation of recall. We failed to replicate that finding. We further conduct a meta-analysis of all replication attempts of these experiments and find that the average effect size (d 0.04) is no different from 0. We discuss some reasons for differences between the results in this article and those presented in Bem (2011).
Research Notes
The most comprehensive replication and meta-analytic challenge to Bem's retroactive recall paradigm, with the largest combined sample of any single replication effort. Discusses researcher degrees of freedom as a possible explanation for original positive findings. A cornerstone of Controversy #2 (Bem FTF) and the broader replication crisis.
Across seven experiments (N=3,289), the retroactive facilitation of recall paradigm from Bem's (2011) Experiments 8 and 9 was replicated using computer-standardized delivery, predetermined sample sizes, and no data inspection before stopping. Six of seven experiments found no evidence of precognition; the combined effect was d≈0.01 with Bayesian BF=70.48 providing 'extreme' support for the null. A meta-analysis of all 19 known replication attempts (N=4,091) yielded an overall effect of d=0.04, 95% CI [-0.00, 0.09], indistinguishable from zero. The only significant moderator was whether Bem himself conducted the experiment (d=0.29 vs. d=0.02 for all others).
Links
Related Papers
Replication Of
Cites
- Why Psychologists Must Change the Way They Analyze Their Data: The Case of Psi — Wagenmakers, Eric-Jan (2011)
- Meta-Analysis of Free-Response Studies, 1992–2008: Assessing the Noise Reduction Model in Parapsychology — Storm, Lance (2010)
- Extraordinary Claims Require Extraordinary Evidence: The Case of Non-Local Perception, A Classical and Bayesian Review of Evidences — Tressoldi, Patrizio E (2011)
- False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant — Simmons, Joseph P (2011)
Companion
- Failing the Future: Three Unsuccessful Attempts to Replicate Bem's 'Retroactive Facilitation of Recall' Effect — Ritchie, Stuart J (2012)
- Results from a Confirmatory Replication Study of Bem (2011): Precognitive Detection of Erotic Stimuli? — Wagenmakers, Eric-Jan (2012)
- Fearing the Future of Empirical Psychology: Bem's (2011) Evidence of Psi as a Case Study of Deficiencies in Modal Research Practice — LeBel, Etienne P (2011)
- Back from the Future: Parapsychology and the Bem Affair — Alcock, James E (2011)
- Bem's 'Feeling the Future' (2011) Five Years Later: Its Impact on Scientific Literature — Silva, Bruno A (2017)
- Raising the value of research studies in psychological science by increasing the credibility of research reports: the transparent Psi project — Kekecs, Zoltan (2023)
Meta Analyzed By
More in Skeptical
Cognitive Styles and Psi: Psi Researchers Are More Similar to Skeptics Than to Lay Believers
Searching for the Impossible: Parapsychology's Elusive Quest
False-Positive Effect in the Radin Double-Slit Experiment on Observer Consciousness as Determined with the Advanced Meta-Experimental Protocol
Cross-Examining the Case for Precognition: Comment on Mossbridge and Radin (2018)
N,N-Dimethyltryptamine and the Pineal Gland: Separating Fact from Myth
📋 Cite this paper
Galak, Jeff, LeBoeuf, Robyn A, Nelson, Leif D, Simmons, Joseph P (2012). Correcting the Past: Failures to Replicate Psi. Journal of Personality and Social Psychology. https://doi.org/10.1037/a0029709
@article{galak_2012_correcting,
title = {Correcting the Past: Failures to Replicate Psi},
author = {Galak, Jeff and LeBoeuf, Robyn A and Nelson, Leif D and Simmons, Joseph P},
year = {2012},
journal = {Journal of Personality and Social Psychology},
doi = {10.1037/a0029709},
}