📌 Appears in:
Plain English Summary
This blockbuster paper — over 35,000 citations — argued that most published scientific results are wrong, kicking off the "replication crisis." Using straightforward math, Ioannidis showed a finding's trustworthiness depends on study size, effect strength, analytical wiggle room, and how plausible the hypothesis was beforehand. The uncomfortable punchline? Well-designed clinical trials get it right about 85% of the time, but underpowered exploratory studies are true only 12-23% of the time. This framework became the go-to toolkit for skeptics questioning psi research, since psi effects tend to be small with limited samples — exactly the recipe Ioannidis warns about.
Abstract
There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias. In this essay, I discuss the implications of these problems for the conduct and interpretation of research.
Links
Related Papers
Cited By
- Why Psychologists Must Change the Way They Analyze Their Data: The Case of Psi — Wagenmakers, Eric-Jan (2011)
- Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling — John, Leslie K (2012)
- Power failure: why small sample size undermines the reliability of neuroscience — Button, Katherine S (2013)
- Can Parapsychology Move Beyond the Controversies of Retrospective Meta-Analyses? — Kennedy, J.E (2013)
- Bayesian and Classical Hypothesis Testing: Practical Differences for a Controversial Area of Research — Kennedy, J.E (2014)
- Testing for Questionable Research Practices in a Meta-Analysis: An Example from Experimental Parapsychology — Bierman, Dick J (2016)
- Evidence for Anomalistic Correlations Between Human Behavior and a Random Event Generator: Result of an Independent Replication of a Micro-PK Experiment — Walach, Harald (2020)
- Planning Falsifiable Confirmatory Research — Kennedy, James E (2024)
- An Agenda for Purely Confirmatory Research — Wagenmakers, Eric-Jan (2012)
- Why Science Is Not Necessarily Self-Correcting — Ioannidis, John P.A (2012)
Companion
- False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant — Simmons, Joseph P (2011)
- Estimating the Reproducibility of Psychological Science — Open Science Collaboration (2015)
- The Garden of Forking Paths: Why Multiple Comparisons Can Be a Problem, Even When There Is No "Fishing Expedition" or "P-Hacking" and the Research Hypothesis Was Posited Ahead of Time — Gelman, Andrew (2013)
- Commentary: Reproducibility in Psychological Science: When Do Psychological Phenomena Exist? — Heino, Matti T. J (2017)
- The "File Drawer Problem" and Tolerance for Null Results — Rosenthal, Robert (1979)
More in Methodology
Paranormal belief, conspiracy endorsement, and positive wellbeing: a network analysis
Addressing Researcher Fraud: Retrospective, Real-Time, and Preventive Strategies — Including Legal Points and Data Management That Prevents Fraud
Quantum Aspects of the Brain-Mind Relationship: A Hypothesis with Supporting Evidence
Paranormal beliefs and cognitive function: A systematic review and assessment of study quality across four decades of research
Experimental evidence of non-classical brain functions
📋 Cite this paper
Ioannidis, John P.A (2005). Why Most Published Research Findings Are False. PLoS Medicine. https://doi.org/10.1371/journal.pmed.0020124
@article{ioannidis_2005_false,
title = {Why Most Published Research Findings Are False},
author = {Ioannidis, John P.A},
year = {2005},
journal = {PLoS Medicine},
doi = {10.1371/journal.pmed.0020124},
}