Opinion: Learning from Peer Review
The grant-review process plays significant roles in the education of researchers and in shaping scientific progress.
By David Irwin, Stephen A. Gallo, and Scott R. Glisson | May 24, 2013
Absent from the many analyses and discussions of scientific peer review are two intangible but very important byproducts: 1) feedback to the applicant and 2) exposure of the reviewers to new hypotheses, techniques, and approaches. Both of these phenomena have a virtual mentoring effect that helps move science forward. Such learning can occur as a consequence of both manuscript review and grant application review, but the review of grant applications, by its very nature, is more iterative and impacts the direction in which research moves very early in the investigation.
The primary mission of reviewing grant applications is to identify the strongest research proposals based on intrinsic scientific merit, hypotheses posed, and feasibility of the approach. At the very least, reviewers will identify the strengths and weaknesses of the application, usually coupled with a scientific merit score, which are then communicated back to the applicant. Thus begins a subtle dialogue between reviewer and “reviewee.”
The simplest form of review is done online, with reviewers sharing their critiques with the applicants electronically. Sometimes, online reviews can include a “chat” or conferral period during which the two (or sometimes three) reviewers may discuss their initial critiques and scores, and then refine either or both to reflect the exchange.
More in-depth reviews can involve regularly convened panels of reviewers who meet over a period of a day or two to review batches of applications grouped by scientific area, learning from each other throughout the process and eventually returning the fruits of their labor back to the applicants in the form of a critique of their proposals. In some of the intramural reviews conducted by the American Institute of Biological Sciences (AIBS), reviews can even involve the applicants directly, in addition to the reviewers and program staff, through an interactive, mid-review teleconference, during which applicants can respond to recommendations and comments in the initial critiques.
In the most elaborate review process, typically reserved for large and/or complex proposals, the review includes a site visit to the PI’s laboratory at the host institution. The review team is typically a group of established scientists and scientist-physicians who have considerable experience and can provide valuable insights and suggestions to the applicant group. Though the setting and procedures are formal and the review groups are not, strictly speaking, advisory groups, there is often an intimate dialogue among applicants and reviewers that can shape and improve the applicants’ research program. Indeed, a survey conducted in 2012 of AIBS reviewers indicated that close to 70 percent believed that feedback to PIs of submitted applications contributed strongly to moving the scientific field forward.
