SuggestedSources

This response was generated by Gemini (gemini-2.5-pro-preview-06-05) using the provided sources.

← Back to Question README ← Back to Index

Want to add more sources or suggest new questions? Contribute on GitHub

What is the replication crisis?

The replication crisis, also known as the reproducibility crisis, is a methodological crisis affecting various scientific fields, where a significant portion of published research findings are difficult or impossible to reproduce in subsequent studies [1, 6]. The crisis first gained widespread attention in psychology and social sciences but is now recognized as a broader issue that also impacts medicine and other disciplines [1, 3, 6]. It undermines the reliability of the scientific record, wastes research funding, and can have severe real-world consequences [3, 5].

Key Evidence and Scope

The crisis was starkly illustrated by a large-scale 2015 study published in Science, which attempted to replicate 100 prominent findings from top psychology journals. The project found that only 36% of the replication attempts produced statistically significant results consistent with the original studies [1]. This low rate of reproducibility suggested that a substantial part of the published literature might not be reliable [1, 2].

While psychology was an early focus, the problem is not isolated to that field. The crisis extends to critical areas of medical science:

Underlying Causes

The replication crisis is not attributed to a single cause but rather a combination of systemic issues, methodological shortcomings, and, in some cases, outright misconduct [7].

  1. Publication Bias: Journals have historically shown a strong preference for publishing novel, positive, and counterintuitive results over negative or null findings. This creates a distorted view of the evidence, as failed replications or studies that find no effect often go unpublished [7].
  2. Low Statistical Power: Many studies are conducted with small sample sizes, which means they lack the statistical power to reliably detect a true effect if one exists. Small, underpowered studies are more likely to produce false positives (finding an effect that isn’t there) or inflated effect sizes [7].
  3. Questionable Research Practices (QRPs): These are methodological gray areas that can lead to false positives. A practice known as “p-hacking” involves analyzing data in various ways until a statistically significant result (a p-value below .05) is found, which is then presented as if it were the original hypothesis. These practices increase the rate of false discoveries in the literature [7].
  4. Scientific Fraud: In the most serious cases, findings are based on deliberately fabricated or falsified data. Such misconduct not only misleads other scientists but can have devastating real-world impacts, as seen in cases of medical fraud that have led to patient harm and death [3, 5, 6].

A Case Study: Stereotype Threat

The concept of “stereotype threat”—the idea that awareness of a negative stereotype about one’s group can impair performance—is a prominent example of a theory caught in the replication crisis. Once a cornerstone of social psychology, the theory has faced a “reckoning” after numerous replication attempts failed to consistently reproduce the original, strong effects. This has forced researchers to re-evaluate the theory’s robustness and the conditions under which it might apply, illustrating how even widely accepted ideas can be challenged by replication failures [4].

Consequences of the Crisis

The inability to trust published research has severe consequences:

Ultimately, the replication crisis is a systemic challenge forcing many scientific fields to improve their methods, promote transparency, and reform publishing incentives to ensure that scientific knowledge is built on a more reliable and trustworthy foundation [1, 7].

Sources

[1] A foundational 2015 study that provided large-scale, quantitative evidence for the replication crisis in psychology, showing that a majority of attempted replications of prominent studies failed to reproduce the original findings. https://pubmed.ncbi.nlm.nih.gov/26315443/

[2] This author interprets data on false discovery rates and replication failures to argue that a high percentage—potentially around 75%—of claims in fields like psychology are likely false due to systemic methodological issues and biases. https://unsafescience.substack.com/p/75-of-psychology-claims-are-false

[3] This op-ed argues that a high-profile case of alleged fraud in Alzheimer’s research demonstrates how scientific misconduct can derail an entire field for decades, wasting billions of dollars and delaying progress on a cure. https://www.nytimes.com/2025/01/24/opinion/alzheimers-fraud-cure.html

[4] A paper by a social psychologist reflecting on how a famous and influential theory, “stereotype threat,” has faced a scientific “reckoning” due to repeated failures to replicate, illustrating the real-world impact of the crisis on established knowledge. https://www.factfaq.com/resources/Revisiting_Stereotype_Threat_-_by_Michael_Inzlicht.pdf

[5] This article argues for treating serious scientific fraud as a crime, highlighting cases where scientific lies, particularly in medicine, have led to a “staggering death toll” and immense human suffering. https://www.vox.com/future-perfect/368350/scientific-research-fraud-crime-jail-time

[6] This investigative report in Nature details the widespread and growing problem of faked or flawed data in clinical trials, arguing that it poses a serious threat to medical knowledge and patient safety. https://www.nature.com/articles/d41586-023-02299-w

[7] This meta-research paper analyzes trends in psychology from 1975-2017, concluding that while statistical power has increased, issues like publication bias remain, which contribute to a high “false discovery rate” and help explain the persistence of the replication crisis. https://pmc.ncbi.nlm.nih.gov/articles/PMC10581498/