A Proposal and Challenge for Proponents and Skeptics of Psi
📄 Original study ↗📌 Appears in:
Plain English Summary
Years before pre-registration became a buzzword in science, Kennedy proposed something bold: treat psi research the way we treat drug trials. He pointed out a damning pattern in psi meta-analyses -- 70-80% of the individual studies pooled together were actually non-significant, and bigger studies weren't producing stronger statistical signals, which breaks a basic rule of how statistical evidence should work. His fix? Assemble a mixed committee of believers, moderate skeptics, and statisticians to agree on study designs before running them. Then he threw down a double challenge: skeptics must say what evidence would convince them, and proponents must show their effects are reliable enough to survive such planning. It was a remarkably fair gauntlet that anticipated where all of science eventually headed.
Actual Paper Abstract
Pharmaceutical research provides a useful model for doing convincing research in situations with intense, critical scrutiny of studies. The protocol for a "pivotal" study that is used for decision-making is reviewed by the FDA before the study is begun. The protocol is expected to include a power analysis demonstrating that the study has at least a .8 probability of obtaining significant results with the anticipated effect size, and to specify the statistical analysis that will determine the success of the experiment, including correction for multiple analyses. FDA inspectors often perform audits of the sites where data are collected and/or processed to verify the raw data and experimental procedures. If parapsychological experiments are to provide convincing evidence, power analyses should be done at the planning stage. A committee of experienced parapsychologists, moderate skeptics, and a statistician could review and comment on protocols for proposed "pivotal" studies in an effort to address methodological issues before rather than after the data are collected. The evidence that increasing sample size does not increase the probability of significant results in psi research may prevent the application of these methods and raises questions about the experimental approach for psi research.
Research Notes
Anticipated the pre-registration movement by nearly a decade. Central to the meta-debate controversy (#10) — argues that meta-analytic approaches alone cannot resolve the psi debate without prospective planning, power analysis, and agreed-upon protocols. Links Kennedy's broader research program on psi's capricious nature.
Pharmaceutical clinical trials, with FDA-mandated power analyses, pre-specified primary outcomes, and independent protocol review, offer a rigorous model for psi research trapped in cycles of inconclusive meta-analysis. Examining published psi meta-analyses reveals that 70-80% of constituent studies are non-significant and z scores fail to increase with sample size — violating a fundamental assumption of statistical testing. A committee of parapsychologists, moderate skeptics, and a statistician is proposed to review pivotal study protocols prospectively. Kennedy challenges skeptics to specify adequate protocols in advance and proponents to demonstrate psi effects are reliable enough for prospective planning.
Related Papers
Cites
- Does Psi Exist? Replicable Evidence for an Anomalous Process of Information Transfer — Bem, Daryl J (1994)
- Consciousness Interactions with Remote Biological Systems: Anomalous Intentionality Effects — Braud, William G (1991)
- A Joint Communiqué: The Psi Ganzfeld Controversy — Hyman, Ray (1986)
- Methods for Investigating Goal-Oriented Psi — Kennedy, J.E (1995)
- The Capricious, Actively Evasive, Unsustainable Nature of Psi: A Summary and Hypotheses — Kennedy, J.E (2003)
- Does Psi Exist? Lack of Replication of an Anomalous Process of Information Transfer — Milton, Julie (1999)
Cited By
Same Research Program
Also by these authors
More in Methodology
Paranormal belief, conspiracy endorsement, and positive wellbeing: a network analysis
Addressing Researcher Fraud: Retrospective, Real-Time, and Preventive Strategies — Including Legal Points and Data Management That Prevents Fraud
Quantum Aspects of the Brain-Mind Relationship: A Hypothesis with Supporting Evidence
Paranormal beliefs and cognitive function: A systematic review and assessment of study quality across four decades of research
Experimental evidence of non-classical brain functions
📋 Cite this paper
Kennedy, J.E (2004). A Proposal and Challenge for Proponents and Skeptics of Psi. Journal of Parapsychology.
@article{kennedy_2004_proposal,
title = {A Proposal and Challenge for Proponents and Skeptics of Psi},
author = {Kennedy, J.E},
year = {2004},
journal = {Journal of Parapsychology},
}