This response was generated by ChatGPT (o3) using the provided sources.
| ← Back to Question README | ← Back to Index |
Want to add more sources or suggest new questions? Contribute on GitHub
Answer
Definition
The “replication crisis” is the growing realization that a surprisingly large share of published scientific results fail to hold up when independent researchers try to repeat (replicate) the study or to re-analyse the data (reproduce) the findings. A failed replication does not prove the original claim is false, but it signals that the original evidence was weaker, noisier, or more error-prone than believed.
How the problem came to light
• Psychology. In 2015 the Open Science Collaboration repeated 100 high-profile psychology experiments; only 36 % yielded a statistically significant effect in the same direction as the original, versus 97 % in the first reports [1].
• Meta-analyses now estimate that roughly three-quarters of new psychology findings are likely false positives or grossly exaggerated [2][7].
• Similar alarms have been raised in biomedicine, from Alzheimer’s disease images later alleged to be fabricated [3] to widespread concerns that 20–50 % of clinical-trial papers are “untrustworthy” because of data or methodological flaws [6].
Why so many results do not replicate
a) Low statistical power – small samples generate unstable estimates and high false-positive rates [7].
b) Publication bias – journals prefer novel, positive results; null findings and replications go unpublished.
c) Questionable research practices – p-hacking, HARKing (hypothesising after results are known), selective reporting.
d) Fraud – a minority of cases are outright fabrication or plagiarism, but high-profile scandals magnify mistrust [3][5].
Consequences
• Waste of money and effort when later studies or clinical trials chase non-existent effects.
• Erosion of public trust in science.
• Potential harm to patients and the public when medical or policy decisions rest on unreliable evidence [5][6].
Reform efforts
• Preregistration of hypotheses and analysis plans.
• Registered Reports in journals (results-blind peer review).
• Data and code sharing to allow re-analysis.
• Larger, multi-lab replication projects (e.g., ManyLabs).
• Statistical education that emphasises effect sizes, confidence intervals, and Bayesian approaches instead of binary p-value thresholds.
Present status
The crisis is now acknowledged across disciplines. Psychology has been the “canary in the coal mine,” but medicine, economics, cancer biology, and social sciences face similar problems. While reforms are spreading, routine replication remains rare, so the full scale of the problem—and whether reforms are working—remains an open empirical question.
Sources
[1] Science 2015: Large multi-lab effort found only 36 % of 100 psychology studies replicated (https://pubmed.ncbi.nlm.nih.gov/26315443/) – concludes reproducibility is “lower than expected.”
[2] Unsafe Science Substack 2023: Argues ≈75 % of psychology claims are false, summarising recent meta-analyses (https://unsafescience.substack.com/p/75-of-psychology-claims-are-false).
[3] New York Times 2025: Describes alleged image manipulation in landmark Alzheimer’s papers and its chilling effect on the field (https://www.nytimes.com/2025/01/24/opinion/alzheimers-fraud-cure.html).
[4] Michael Inzlicht pdf: Reviews failures to replicate “stereotype threat,” calling for soul-searching in social psychology (https://www.factfaq.com/resources/Revisiting_Stereotype_Threat_-_by_Michael_Inzlicht.pdf).
[5] Vox 2023: Explains human and economic costs of scientific fraud and why punishments are rare (https://www.vox.com/future-perfect/368350/scientific-research-fraud-crime-jail-time).
[6] Nature 2023: Reports that 20–50 % of published randomised trials appear untrustworthy, urging systematic audits (https://www.nature.com/articles/d41586-023-02299-w).
[7] PLOS One 2023: Analyses 44 years of psychology papers, estimating a 65–75 % false-discovery rate and chronic low power (https://pmc.ncbi.nlm.nih.gov/articles/PMC10581498/).