| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Irogers{at}ajroffice.org
So how do you identify a good reviewer? It's not easy to predict the quality of reviews on the basis of past academic productivity or present academic reputation; in fact, no definite correlation exists. Reviewing manuscripts is a form of criticism and some people are simply better at it than others. Music critics are not necessarily good musicians, nor are drama critics necessarily good actors. And so it is with academicians. Some academicians have a knack for reviewing, whereas others don't. One reviewer may not be sufficiently committed to the review process and thus performs the task perfunctorily. Another would-be reviewer, even one with a fine academic reputation, may be hesitant or unwilling to criticize and judge the work of others.
So, in the end, how do you identify a good reviewer? It is rather straightforward. Send out a manuscript and see how the reviewer does. A good reviewer is identified by the high quality of the reviews performed.
Peer review serves several purposes. First, it assesses whether a manuscript contains information that is original, timely, and important. Second, peer review determines whether the research has been properly performed. Third, peer review identifies areas in which the research and resultant manuscript could be improved. And, fourth, peer review advises the editor about whether the manuscript should be accepted for publication.
Foremost, a good review addresses these questions: "Does this paper contain new information?" "Is there proof?" "Is there a gold standard?" "Was the case selection proper?" "Were the statistical methods appropriate?" "What should be done to improve the manuscript?"
Reviewers should point out confusing phrases and sentences, especially those in which the authors' intent or meaning is unclear, and request clarification. If possible, reviewers should suggest changes in the text that help clarify meaning. It is more important to identify statements that lack clarity than it is to correct misspellings or errors in syntax and grammar. At the AJR, professional manuscript editors will subsequently review all accepted manuscripts; misspellings or errors in syntax and grammar are best left to them.
Another duty of reviewers is to comment on the use of tables as well as the appropriateness of figures and references. Reviewers should ask themselves: "Could certain tables or figures be deleted without compromising the article?" "Have the key points been illustrated?" "Should additional figures be included?" "Are the references up-to-date and appropriate, or have key references been overlooked?"
Finally, each reviewer writes (on plain paper! no letterhead, please) a narrative review that should conclude by recommending to the editor the editorial action to be taken on the manuscript: accept, reject, or reject with the opportunity to revise and resubmit. The reviewer then completes a score sheet (Appendix 1, next page) that summarizes the review in numeric form. (The AJR manuscript review score sheet was extensively improved by Mark Kliewer during his time in the Editorial Office as a Figley Fellow.)
The AJR obtains two reviews on all manuscripts except case reports and selected technical notes, both of which initially receive only one review. Ten percent to 15% of manuscripts require a third review before an editorial decision can be reached.
For the majority of manuscripts, reviewers agree on the merits. For these manuscripts, overall rankings fall within one or two points of each other on our 10-point scale. In slightly fewer than 10% of manuscripts, there is a divergence of opinion about overall ranking between the two reviewers: five or more points of divergence on our 10-point scale. One reviewer says to publish; the other says to reject. In this situation, we seek the opinion of a third reviewer. Originally, I believed that third reviewers would side with one or the other of the previous reviewers—but, alas, that rarely occurs. What usually happens is that the third reviewer comes down someplace between the two previous reviewers, which then usually leads to rejection with the opportunity to resubmit. When resubmitted, the manuscript is returned to the original reviewers for another review. This assures us that the reviewers' concerns have been addressed and that the requested substantive revisions have been made.
At the AJR, editors peruse a submitted manuscript before assigning reviewers. Such assignments are based on the reviewers' interest and expertise as they relate to the subject of the manuscript. References in a submitted manuscript are often used to determine if a reviewer for the AJR has written on the subject. If so, that reviewer is likely to be assigned the review.
When reviews are returned to the AJR, an editor grades each review, and the AJR database calculates and records the days taken to perform the review. Also, the AJR database calculates a cumulative average of each reviewer's grades and the average days taken to perform manuscript reviews. These data are also used by the AJR editors when selecting reviewers. Better reviewers are clearly identified by higher average scores on reviews and shorter average times to review. We try not to ask a reviewer to perform more than one review a month, if at all possible. Editors don't want to burn out good reviewers. But admittedly, in our desire to obtain a good review, AJR editors push the envelope a bit, particularly with our better reviewers.
And what do peer reviewers receive for all their efforts? Recognition and thanks for a job well done. Reviewing is a worthwhile and noteworthy contribution to your scientific colleagues and to scientific journals. By serving as a reviewer, an academician can stay abreast of the latest developments in specific fields of interest and, at the same time, acquire insights into scientific writing and research that will surely improve one's own academic and scientific endeavors.
We invite your participation. However, the bad news here is that because of cost and time constraints to post paper manuscripts for review, AJR reviewers must reside in either the United States or Canada. The good news is that computer software is under development to facilitate the electronic transmission of manuscripts and figures, which will save both time and money. When the entire editorial process goes online, the AJR will be able to call on the expertise of reviewers worldwide.
Until then, if you can't review for the AJR, volunteer to review for journals closer to home. You will be glad you did.
APPENDIX I. AJR Manuscript Review Score Sheet
|
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
This article has been cited by other articles:
|
|
 |
|
  R. G. Sheiman The RSNA Reviewer Mentorship Program Radiology, September 1, 2007; 244(3): 631 - 632. [Full Text] [PDF]
|
|
|
|
 |
|
 M. A. Kliewer, K. S. Freed, D. M. DeLong, P. J. Pickhardt, and J. M. Provenzale Reviewing the Reviewers: Comparison of Review Quality and Reviewer Characteristics at the American Journal of Roentgenology Am. J. Roentgenol., June 1, 2005; 184(6): 1731 - 1735. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. E. Spieth Peer Reviewers: Reviewing Manuscripts Am. J. Roentgenol., December 1, 2002; 179 (6): 1645 - 1645. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
