Mindless Statistics
📄 Original study📌 Appears in:
Plain English Summary
Here's a shocking fact: about 90% of psychology students, teachers, and even professors misunderstand what p-values actually mean. Gigerenzer reveals that the standard way scientists test their hypotheses — the 'null ritual' — is actually a Frankenstein mashup of two different statistical frameworks that neither original inventor would recognize or endorse. It gets worse: statistical power (basically, your experiment's ability to detect a real effect) has averaged a coin-flip-level 50% for decades with zero improvement. This matters hugely for parapsychology debates, because without proper power analysis, a failed replication tells you almost nothing. You can't use a blunt tool to make a sharp argument. Gigerenzer pushes for a richer toolkit including Bayesian methods, effect sizes, and exploratory analysis instead of robotically chasing p-values.
Actual Paper Abstract
Statistical rituals largely eliminate statistical thinking in the social sciences. Rituals are indispensable for identification with social groups, but they should be the subject rather than the procedure of science. What I call the "null ritual" consists of three steps: (1) set up a statistical null hypothesis, but do not specify your own hypothesis nor any alternative hypothesis, (2) use the 5% significance level for rejecting the null and accepting your hypothesis, and (3) always perform this procedure. I report evidence of the resulting collective confusion and fears about sanctions on the part of students and teachers, researchers and editors, as well as textbook writers.
Research Notes
Foundational methodological critique with direct implications for parapsychology debates: shows why p-values alone cannot confirm or refute psi effects, why failure to replicate is uninformative without power analysis, and why post-hoc significance testing is epistemically hollow. Kennedy's psi methodology series builds on this tradition.
The 'null ritual' — NHST as routinely practiced in psychology — is an incoherent hybrid of Fisher's null hypothesis testing and Neyman-Pearson decision theory that neither statistician endorsed. Surveys show ~90% of psychology students, teachers, and professors hold false beliefs about p-values (Haller & Krauss, 2002). Meehl's conjecture that null hypotheses in large non-experimental samples are virtually always false is empirically supported: 46% of random directional predictions confirmed as significant across 81,000+ MMPI-2 responses (Waller, 2004). Statistical power in psychology has averaged ~50% for medium effects since 1962 without improvement. Advocates replacing the null ritual with a toolbox including Bayesian methods, effect sizes, and exploratory analysis.
Links
Related Papers
Companion
- Can Parapsychology Move Beyond the Controversies of Retrospective Meta-Analyses? — Kennedy, J.E (2013)
- Is the Methodological Revolution in Psychology Over or Just Beginning? — Kennedy, J.E (2016)
- Planning Falsifiable Confirmatory Research — Kennedy, James E (2024)
- Editorial: Emerging Research: Self-Ascribed Parapsychological Abilities — Simione, Luca (2025)
- Statistical Problems in ESP Research — Diaconis, Persi (1978)
- Power failure: why small sample size undermines the reliability of neuroscience — Button, Katherine S (2013)
- A Practical Solution to the Pervasive Problems of p Values — Wagenmakers, Eric-Jan (2007)
- Feeling the Future: Experimental Evidence for Anomalous Retroactive Influences on Cognition and Affect — Bem, Daryl J (2011)
- Why Psychologists Must Change the Way They Analyze Their Data: The Case of Psi — Wagenmakers, Eric-Jan (2011)
More in Methodology
Paranormal belief, conspiracy endorsement, and positive wellbeing: a network analysis
Addressing Researcher Fraud: Retrospective, Real-Time, and Preventive Strategies — Including Legal Points and Data Management That Prevents Fraud
Quantum Aspects of the Brain-Mind Relationship: A Hypothesis with Supporting Evidence
Paranormal beliefs and cognitive function: A systematic review and assessment of study quality across four decades of research
Experimental evidence of non-classical brain functions
📋 Cite this paper
Gigerenzer, Gerd (2004). Mindless Statistics. The Journal of Socio-Economics. https://doi.org/10.1016/j.socec.2004.09.033
@article{gigerenzer_2004_mindless_statistics,
title = {Mindless Statistics},
author = {Gigerenzer, Gerd},
year = {2004},
journal = {The Journal of Socio-Economics},
doi = {10.1016/j.socec.2004.09.033},
}