When the Findings Refused to Repeat
In the early 2010s, psychology and the adjacent sciences entered a reckoning that has since reshaped how every careful reader approaches every scientific claim: the replication crisis, the discovery, through the systematic large-scale replication attempts, that a shocking fraction of the published findings could not be reproduced by independent researchers following the original protocols. Psychology's own large replications (the Open Science Collaboration's 2015 effort reproducing fewer than half of the hundred studies attempted, the effect sizes halving on average in the replications) made the crisis famous, but the problem's roots and its reach extend through the biomedical sciences this series depends on (the preclinical findings that fail to translate, the animal studies that the industry could not reproduce, the clinical signals that the larger trials contradict), and the crisis's lessons (the publication bias, the p-hacking, the small-sample noise, the incentive structures that produced the unreliability) are the series' epistemology in its starkest form: the difference between the finding and the truth, and the machinery that determines which is which. This post is the replication crisis account: what was discovered, why it happened, and what it means for the drug science this series has reported.
What Was Discovered
The discoveries deserve their enumeration, because the scale was the shock. The psychology replications (the OSC's roughly 40 percent reproduction rate and the effect sizes' halving, the social-priming and the ego-depletion literatures' collapses that the high-profile cases became), the preclinical reckoning (the pharmaceutical industry's own documented frustration: the internal replication attempts that failed on the majority of the academic findings the drug development relied upon, the cancer-research reproducibility work that the Begley-and-Ellis analyses publicized, the billion-dollar consequence of the unreproduced preclinical findings that the pipelines advanced), the basic-science layer (the cell-line contamination and the antibody-specificity failures and the reagent problems that the systematic checks revealed), and the clinical layer (the single-trial signals that the replications and the meta-analyses contradicted, the series' trial-phase post's attrition funnel being, in part, the replication crisis at the pipeline's scale). The common threads (the small samples that the noise dominates, the flexible analyses that the multiple-comparisons exploit, the publication bias that the positive-results' preference produces, the file-drawer of the unpubished nulls) were the crisis's diagnostic, and the reform movement (the pre-registration, the registered reports, the open data, the replication's prestige-restoration) its response.
Why It Happened
The causes deserve the systemic treatment, because the crisis was not fraud but incentive. The publication system (the positive-results preference that the journals' business models produced, the novelty premium that the replication's low prestige enforced, the career incentives that the publication-counts reward), the statistical naivety (the p-value threshold's tyranny that the study-reading post covered, the multiplicity that the unregistered fishing exploits, the small samples that the funding's scarcity produced and the noise's domination that they invited), and the methodological flexibility (the researcher-degrees-of-freedom that the undisclosed analytic choices provide, the p-hacking's garden-of-forking-paths that the retrospective storytelling produces) combined into the reliability's collapse, and the perverse equilibrium (everyone's rational behavior producing the collective's unreliability, the prisoner's dilemma of the publish-or-perish) meant the correction required the structural reforms rather than the individual virtue.
The Reform Movement
The reforms deserve the catalog, because they are the reader's quality filters. Pre-registration (the trials' and the studies' registration before the data collection, the analysis-plan's commitment that prevents the endpoint-switching and the selective-reporting, the clinical-trial registries that the medical literature now requires and the social-science registries that the crisis produced), registered reports (the journal format that reviews the methods and the analysis plan before the results exist, accepting the paper on the design's merit regardless of the outcome, the publication bias's structural fix), the open-science practices (the data and the code sharing that the independent verification requires, the post-publication review that the openness enables), the statistical reform (the effect-size and the confidence-interval emphasis over the p-value threshold, the multiplicity corrections' enforcement), and the replication's institutionalization (the replication journals and the funding and the prestige that the reform movement built). The drug science's adoption (the FDA's registration requirements, the ICMJE's standards, the pre-registration's norm in the serious clinical literature) is real and incomplete, and the reader's filter is the reform's consumer version: the registered-and-replicated finding outweighs the unregistered-and-unreplicated, the preprint weighed accordingly, the single-study signal discounted by the crisis's base rates.
The Implications for This Series' Field
The crisis's implications for the series' coverage deserve the specific application. The preclinical-translation gap (the BDNF and the critical-period findings that the animal data supports and the human translation awaits, the series' plasticity coverage's discipline being the crisis's lesson applied), the psychedelic-placebo problem (the expectancy-confounded findings that the replication's demand for the active placebos addresses, the series' placebo post's reform agenda), the microdosing literature's collapse (the self-blinding studies' null results as the replication crisis's psychedelic case-study, the belief-tracking-the-experience finding that the crisis's framework predicted), the neuroimaging's vulnerability (the small-sample imaging findings' low reproducibility that the field's meta-science has documented, the default-mode-network literature's specific replication questions), and the kratom-and-survey literature's limits (the observational-tier ceiling that the study-reading post assigned and the replication crisis reinforces) are the applications, and the series' closing counsel deserves the crisis's framing: the epistemology this series has practiced (the evidence hierarchy, the skepticism of the single study, the expectancy's accounting, the pyramid's respect) is the replication crisis's consumer translation, and the reader who has absorbed the series has, in effect, absorbed the crisis's lessons before the post ever named them.
The Bottom Line
The replication crisis was the sciences' own quality-control discovery: the finding that a large fraction of what was published could not be repeated, the causes systemic rather than fraudulent (the publication bias, the statistical naivety, the incentive structures), and the reforms structural (the pre-registration, the registered reports, the open data, the replication's restoration). For the drug science this series reports, the crisis is the standing caution (the preclinical-translation gap, the expectancy-confounded trials, the single-study signals that the attrition funnel and the meta-analysis must test) and the series' epistemology is its consumer version (the skepticism this series has brought to every claim, from the microdosing hype to the press-release amplification, is the replication crisis's discipline practiced one reader at a time). The truth, the crisis taught, is what repeats. The finding is what publishes. The distance between them is the series' whole project, and the reader who holds the distinction holds the only literacy the publishing era has ever required.
