2 June 2014, 2.54pm AEST
Hate the peer-review process? Einstein did too
Most academic papers today are published only after some academic peers have had a chance to review the merits and limitations of the work. This seems like a good idea, but there is a growing movement that wants to retort as Albert Einstein did to such a review process.
Academic review process was different in Einstein’s time. In his brilliant career, the only time his work was subjected to blind peer review – the authors don’t know the reviewers and vice versa – he showed contempt for what is now the gold standard of science. Was Einstein right to be so suspicious of the peer-review process? Should we learn from him and begin to question the widespread use of peer review in academic science?
The first part of Einstein’s career was in the German-speaking world. The German physics journals, in which Einstein published his breakthrough work, didn’t have the same peer-review system we use today.
For instance, the Annalen der Physik, in which Einstein published his four famous papers in 1905, did not subject those papers to the same review process. The journal had a remarkably high acceptance rate (of about 90-95%). The identifiable editors were making the final decisions about what to publish. It is the storied editor Max Planck who described his editorial philosophy as:
To shun much more the reproach of having suppressed strange opinions than that of having been too gentle in evaluating them.
Many of the core scientific discoveries were not peer reviewed to modern standards. For example, the publication of the foundational paper describing the double helical structure of DNA by James Watson and Francis Crick in 1953 would have been jeopardised in the context of the classic review system as we know it, because of its speculative nature.
At the prestigious journal Nature, the peer-review system was only formally introduced in 1967. More recently, the discovery of distortion in gravitational waves by a telescope at Harvard – which has crucial consequences for our understanding of the formation of the universe – was presented as preliminary and treated with extreme caution and even sometimes with denigration, because it had not been peer-reviewed.
American adventure
It was only after Einstein came to the US in 1935 that he came face to face with the peer review process. He and his younger colleague, Nathan Rosen, sent a paper on gravitational waves to Physical Review, a journal which had established its reputation as the premier physics journal in the US. The paper had the potential to be highly controversial as it challenged the idea that gravitation was a wave.
John Tate, the editor of the journal, hesitated over Einstein’s paper for a month. He then send it to a reviewer for comments – his selected reviewer was probably the famously gossipy Howard Percy Robertson, one of Einstein’s colleagues at Princeton. The reviewer returned ten pages of comments which cast doubt on many of the central claims in the paper. The editor returned these comments to Einstein, asked him to consider the issues, and make any changes he saw necessary. Here is how Einstein reacted:
We (Mr. Rosen and I) had sent you our manuscript for publication and had not authorised you to show it to specialists before it is printed. I see no reason to address the – in any case erroneous – comments of your anonymous expert. On the basis of this incident I prefer to publish the paper elsewhere.
Although he withdrew the paper from Physical Review, Einstein went on to publish it in a much more low key outlet, the Journal of the Franklin Institute. However, the published version contains substantial revisions. It appears these revisions were largely on the basis of a discussion he had with Robertson at Princeton. The revised version toned down many of his original huge claims. These revisions may have saved him from public embarrassment.
What would Einstein say today?
Some might see this as an amusing historical incident. But we think it contains some important lessons for scientists of all kinds today. This is because it reflects the current tension regarding the peer-review system.
The story reminds us that double-blind peer review is only a relatively recent invention. For most of history of science, scientific advances were judged in a much more open and public fashion. It also shows us that the peer-review process can provoke displeasure among even the greatest. It can mean scientists not listening to criticism. Sometimes the result is that many ideas don’t see the light of day.
These anecdotal lessons point to wider issues with the peer-review process, which itself hasn’t been studied in much detail. The review process was meant to save scientists from mistakes and public embarrassment. The idea was that peers help to improve our work, and the review process of high-status journals can serve as “stamps of approval” or simply signal of quality.
But sometimes a collegial discussion rather than formalised peer review can be a better way of getting the message across. So far the peer-review process has been largely an item of faith – something that probably produces better science. However, there is a growing body of evidence which is challenging this notion.
An extensive review of the literature on peer review in 1998 identified problems. They found that there is a low level of reliability and agreement over the quality of submitted papers, largely because of a lack of objective evaluation criteria. Even worse, reviewers make mistake in their evaluation and often accept papers they should have rejected. As a direct consequence, established journals are usually biased against innovative work.
In our own field of management science, some have claimed the peer-review system means academic work can simply end up losing its integrity during the review process, and can result in trivial and boring research.
On a more positive note, when reviews are perceived of quality by authors, they tend to generate more citations, which is a measure of the number of times a research paper is mentioned in other journals and is considered a mark of quality. Also, reliability is not necessary for an efficient review process – often it is the process of peer review itself that contributes to improving the paper. Reviewers play a developmental role in the construction of knowledge, and the energy they deploy in this process is primarily driven by moral motives rather than any material interest.
...
Read more here/Leia mais aqui: The Conversation
+++++
Read more here/Leia mais aqui: The Conversation
+++++
NOTA DESTE BLOGGER:
Hoje, uma das funções da revisão por pares é de guarda-cancelas: todos e quaisquer artigos contra o paradigma aceito pela Nomenklatura científica, não são aprovados para publicação. Mesmo que as evidências apoiem o autor do artigo!
