Replication Crisis

The replication crisis is among the most devastating recent failures in science.

Manuscripts may add a throw-away line in their limitations section that we should replicate, but most researchers still don't replicate their own work or the work of others. Many researchers still use inadequate sample sizes and most still fail to do a priori power analyses. It is still rare that researchers openly share their data and ever rarer that they openly share their code in their analysis scripts.

Isn't this just science working as it should?

No.

The corrective mechanisms in science have discovered the error and raised the alarm.
While the alarm has been raised, the corrective mechanisms have not fixed the system. The system was broken and still remains broken.

I suppose we could say that science is "working as it should" to the same degree that an internal affairs investigation revealing police corruption means law enforcement is "working as it should": the system has a corrective mechanism, which raised the alarm. That shows that the system is working, right?

Wrong.

There have been very few repercussions.
Researchers that published invalid studies still hold their prestigious positions. There were no mass firings or even mass retractions over p-hacked research. Researchers that published HARKed papers have not been suspended or put on academic probation. Researchers that published non-replicable findings have not been subject to fines or any other sort of reprimand. Most scientists that published invalid research have not even faced public scrutiny or outrage, with a few major cases of fraud being the notable exception, at least for the short duration of their news-cycle. Journals haven't even marked non-replicable research as dubious to let new readers know that there have been subsequent failures to replicate.

Many researchers are still doing dubious research, many reviewers are still accepting dubious research, and many editors are still publishing dubious research.

Index

Return to Psychology The State of the Field

Continue to Readings - Crises in our field

Jump to Undergraduate Psychology