Boston Globe challenges peer review system
Today's Boston Globe questions the value of the peer review system, in which other researchers critique scientific papers before they are published (Flaws are found in validating medical studies). The Globe cites the following problems:
- Reviewers are anonymous --so readers don't know who they are
- Reviewers' comments are kept confidential --so it's impossible for outsiders to review their quality
- Reviewers are unpaid --so may not take it seriously
- Standards vary --so "peer reviewed" means different things depending on the journal
As evidence that there is a problem, the Globe cites a study that shows up to one-third of journal articles are eventually "contradicted or seriously questioned."
New England Journal of Medicine editor, Dr. Jeffrey Drazen can usually be counted on to provide the Globe with a reactionary quote in response to changes in the publishing model, and he fulfills his role again today:
"We don't think the system is broken and needs to be overhauled."
The Globe article raises some important points, but my assessment is somewhat different:
- It's not a problem that a fairly high percentage of studies are later contradicted or challenged. If only the most airtight papers were published a lot of very good ones would never see the light of day, and researchers would be slower to challenge the status quo. Meanwhile we can also learn from research that later turns out to be wrong or 'questionable'
- It's ironic that journals that defend their high subscription prices by claiming that the cost of editing is so high use unpaid labor for the most critical tasks
- A major problem with the peer review system that the Globe doesn't mention is that using unpaid reviewers slows the process down. It's hard to impose deadlines on a free resource