Replication

Attempts at replication of social science papers showed that half could not be replicated with statistical significance, and re-analysis showed that 2% of the studies even came to the opposite conclusion! Some of the difficulty maybe downstream of the fact that less than 30% of the 600 papers gave enough detail that replication could even be attempted. 

Finally, SCORE checked papers’ replicability — the most onerous of the three tasks. Researchers endeavoured to repeat entire experiments, gathering and analysing the data from scratch. Of the 164 studies that they focused on, they were able to replicate only 49% with statistical significance¹. That figure is roughly in line with the results of other attempts to replicate scientific findings.

Gurwinder concludes from the 7-year long project published in Nature  it’s now wiser to assume a social science study is flawed until there’s reason to believe otherwise. Similar problems persist in biomedical studies.