Skip to main content

Scientific Utopia: II. Restructuring Incentives and Practices to Promote Truth Over Publishability

πŸ“„ Original study β†—
Nosek, Brian A, Spies, Jeffrey R, Motyl, Matt β€’ 2012 Modern Era β€’ methodology

Plain English Summary

Here's a wake-up call for science itself. The authors tried to replicate a splashy psychology finding and -- with nearly 1,300 participants -- got absolutely nothing. That personal experience fueled a deep dive into why science keeps producing results that don't hold up. The culprit? A publish-or-perish culture that rewards exciting, positive findings and buries boring-but-honest null results. They catalogue nine sneaky tricks (called questionable research practices) that researchers use, often without realizing it, like peeking at data early and stopping when it looks good, or only reporting the analyses that worked. Their fix is a bold transparency overhaul: share your data openly, pre-register your study plans before collecting data so you can't move the goalposts, and build systems that actually reward replication (re-running studies to verify them). This paper became a rallying cry for the open-science movement and remains the yardstick against which modern parapsychology research is judged.

Abstract

An academic scientist's professional success depends on publishing. Publishing norms emphasize novel, positive results. As such, disciplinary incentives encourage design, analysis, and reporting decisions that elicit positive results and ignore negative results. Prior reports demonstrate how these incentives inflate the rate of false effects in published science. When incentives favor novelty over replication, false results persist in the literature unchallenged, reducing efficiency in knowledge accumulation. Previous suggestions to address this problem are unlikely to be effective. For example, a journal of negative results publishes otherwise unpublishable reports. This enshrines the low status of the journal and its content. The persistence of false findings can be meliorated with strategies that make the fundamental but abstract accuracy motive – getting it right – competitive with the more tangible and concrete incentive – getting it published. We develop strategies for improving scientific practices and knowledge accumulation that account for ordinary human motivations and self-serving biases.

Links

Related Papers

Also by these authors

More in Methodology

πŸ“‹ Cite this paper
APA
Nosek, Brian A, Spies, Jeffrey R, Motyl, Matt (2012). Scientific Utopia: II. Restructuring Incentives and Practices to Promote Truth Over Publishability. Perspectives on Psychological Science. https://doi.org/10.1177/1745691612459058
BibTeX
@article{nosek_spies_motyl_2013_scientific_utopia,
  title = {Scientific Utopia: II. Restructuring Incentives and Practices to Promote Truth Over Publishability},
  author = {Nosek, Brian A and Spies, Jeffrey R and Motyl, Matt},
  year = {2012},
  journal = {Perspectives on Psychological Science},
  doi = {10.1177/1745691612459058},
}