Time to review peer review
17:45 21 June 2012
Andrew Pontzen, astrophysics postdoc
(Image: SelectStock/Getty Images)
Standard lore has it that scientific results are supposed to be published in academic journals before they are even worth discussing. These publications use a "peer-review" system to determine the validity of a paper. If it's not valid in the eyes of the relevant expert community, it won't be published. It's supposed to be a way we can tell good science from bad: with the community as our judge.
That makes some sense but the ideal isn't quite a reality (at least not in my field, theoretical physics and astronomy). We are not really trusting the community; we are trusting one or two selected members of the community known as "the referees". We are trusting the editor of the journal to select referees who are competent and free from competing interests. And we are supposed to put our trust in the process despite the referees being completely anonymous - neither the author nor the reader knows who's involved.
Even if a referee believes the paper is worthy of publication, he or she can demand the author make changes. The author must respond by revising the paper to the referee's satisfaction. The paper bounces back and forth in a slow-motion game of tennis. If the author believes the referee is playing unfairly, any appeals must be made to the journal's editor. But editors rarely undermine a referee that they selected in the first place.
Assuming the work does eventually get published, the author's original intentions are hopelessly mixed up with the biases of anonymous third parties. Genuine, honest scientific disaccord is obscured by a process which is invisible to the reader.
I am not arguing to remove peer review entirely from the scientific publishing process but it does need a radical overhaul, and now is the right time to start. Peer-review offered a quality-control filter in an age where each printed page cost a significant amount of money. It's not totally clear that the quality-control filter was ever particularly effective, but at least it gave journals a way to cut down the volume of print.
These days most physicists now download papers from arxiv.org, a site which hosts papers regardless of their peer-review status. We skim through the new additions to this site pretty much every day, making our own judgements or talking to our colleagues about whether each paper is any good. Peer-review selection isn't a practical priority for a website like arxiv.org, because there is little cost associated with letting dross rot quietly in a forgotten corner of the site. Under a digital publication model, the real value that peer review could bring is expert opinion and debate; but at the moment, the opinion is hidden away or muddled up because we're stuck with the old-fashioned filtration model.
...
Read more here/Leia mais aqui: New Scientist
