Tuesday, June 02, 2026

Hidden in the Replication Crisis

 Nonreplicable Findings are cited more than replicable ones. My cynical self whispered that these are things that people want to be true, evidence be damned, because it would be so cool if it were.  I was therefore pleased to read in the next paragraph that Stewart-Williams calls it Steve's Law, that Boring findings and non-PC findings are more replicable than interesting or PC ones.* The paper's own abstract says something similar

Abstract:  We use publicly available data to show that published papers in top psychology, economics, and general interest journals that fail to replicate are cited more than those that replicate. This difference in citation does not change after the publication of the failure to replicate. Only 12% of postreplication citations of nonreplicable findings acknowledge the replication failure. Existing evidence also shows that experts predict well which papers will be replicated. Given this prediction, why are nonreplicable papers accepted for publication in the first place? A possible answer is that the review team faces a trade-off. When the results are more “interesting,” they apply lower standards regarding their reproducibility.

*Also at the link are odd studies showing that birds are more afraid of women than men - currently unexplained. 

No comments: