| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
1 Department of Radiology, 259 First St., Winthrop-University Hospital, Mineola,
NY 11501.
2 Department of Radiology, School of Medicine, State University of New York at
Stony Brook, Stony Brook, NY 11794.
3 Editorial Office, Radiology, 1001 E. Broad St., Ste. 310, Richmond,
VA 23219.
4 Editorial Office, RadioGraphics, 4520 East-West Hwy., Ste. 630,
Bethesda, MD 20814.
Received March 15, 2002;
accepted after revision May 30, 2002.
Presented at the International Congress on Peer Review in Biomedical
Publication, Barcelona, Spain, September 2001.
Abstract
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
MATERIALS AND METHODS. The editors of two radiology journals reviewed 880 major original manuscripts submitted to their journals during a 6-month period without knowledge of the identities and institutions of the authors. Each manuscript was inventoried for possible author or institutional unblinding and for the specific types of unblinding violations.
RESULTS. Of 880 manuscripts, 300 (34%) contained information that potentially unblinded the identities of the authors, their institutions, or both. The editors correctly identified the authors or institutions in 221 (74%) of the 300 manuscripts, which represented 25% of the total manuscripts. The most frequent unblinding violations were statement of the authors' initials within the manuscript, referencing work "in press," identifying references as the authors' previous work, and revealing the identity of the institution in the figures.
CONCLUSION. Despite explicit instructions to authors, 34% of 880 prospectively evaluated manuscripts submitted to two radiology journals contained information that potentially or definitely unblinded the identities of the authors or their institutions.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
The anecdotal experience of the first author of this study, while reviewing manuscripts, was that authors submitting manuscripts for review to radiology journals often unblind themselves or their institutions. Therefore, the purpose of our study was to prospectively determine the incidence and nature of unblinding by authors as to their identities or their institutions, using a large series of original major manuscripts submitted to two radiology journals with double-blinded peer review policies that are stated in each issue in their publication information for authors.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Each editor reviewed every major original article submitted to his journal for possible publication during a 6-month period, without knowledge of the identities and institutions of the authors. A total of 880 manuscripts were inventoried in a blinded manner by the two editors: 734 at the first journal and 146 at the second journal. When peer review of the manuscript was completed and a decision on the manuscript was pending, editorial staff members gave each manuscript to the editor with a title page that included only the manuscript title. At the first journal, the acknowledgments page was removed before each manuscript was given to the editor. No other alteration of the original manuscript was made by the editorial staff, so that any other unblinding by the authors would be evident. The editorial staff of the first journal attempts to remove blatant unblinding, such as statement of the authors' names, initials, and institutions in the body of the manuscript, before manuscripts are sent to reviewers, but does not fix other forms of unblinding, such as self-referencing.
Each manuscript was inventoried, using a onepage worksheet, for the presence or absence of possible author or institutional unblinding as well as for specific types of unblinding violations. All instances of potential unblinding in a manuscript were recorded on the worksheet. At the first journal, the editor evaluated each manuscript for eight categories of possible unblinding: authors names stated in the manuscript text, authors' initials stated in the manuscript, previous work referenced and stated as such, authors' institution stated in the text, authors' institution stated in the figures, work "in press' referenced as such, inappropriate citation of abstracts (i.e., citation of abstracts relating to the manuscript itself that reveals the authors' identities), and "other." At the second journal, all eight of these categories were used, as were two additional ones: acknowledgments that identify the authors or institutions and reprinted figures that identify the authors or their institutions.
The editors searched each manuscript for evidence of unblinding but did not attempt to guess the identities of the authors or their institutions on any other basis (e.g., personal knowledge of subspecialty research interests by specific groups of investigators or of similar abstracts presented at national or international meetings). If the editor believed that the authors had unblinded their identities or their institutions, their names or institutions were noted on the reverse sides of the worksheets, which were subsequently compared by the editorial office staff at each journal with the unblinded title pages of the manuscripts. The staff then indicated on the front of the worksheets whether the editors had correctly identified the authors or institutions. When data collection was complete, the front sides only of all data sheets were photocopied and sent to the first author of this study, thus maintaining author confidentiality as required by each journal's editorial office policies. The first author then calculated the incidence of unblinding for the manuscripts at each journal, as well as the incidence of each specific type of unblinding and the rate at which each editor correctly identified the authors or institutions when unblinding was thought to have occurred.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Three hundred (34%) of the 880 manuscripts contained one or more author violations of the peer review policy that potentially unblinded the identities of the authors, their institutions, or both. This information was included in 35% (257/734) of manuscripts at the first journal, and 29% (43/146) of manuscripts at the second journal. The two editors correctly identified the authors, their institutions, or both in 74% (221/300) of the manuscripts with violations: 70% (179/257) at the first journal, and 98% (42/43) at the second journal. Thus, unblinding occurred in 25% (221/880) of all manuscripts studied.
We found 374 individual category violations of the blinded peer review policy by authors, or approximately 0.43 per manuscript (0.8 per manuscript of those with violations). Each category violation represented at least one instance of potential unblinding in an individual manuscript in one of the eight (at the first journal) or 10 (at the second journal) categories. Three hundred fifteen violations occurred at the first journal, and 59, at the second journal. Only one category violation occurred in 238 manuscripts, and 62 manuscripts had violations in more than one category. At the first journal, 71 of the 78 manuscripts that were potentially unblinded but subsequently not identified by the editor were instances in which the authors stated their initials in the text of the manuscript but they were not recognized by the editor.
The most frequent violations (Table 1) were the statement of the authors' initials in the body of the manuscript, referencing the authors' own work in press, identifying references as the authors' previous work, revealing the identity of the institution in the figures, and stating the identity of the authors' institutions in the body of the manuscript. In addition, authors stated their names in the manuscript or revealed their identities or those of their institutions through the use of previously published figures or in the acknowledgments. We found one instance of inappropriate citation of abstracts, and 25 miscellaneous other violations.
|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Whether or not reviewers should be blinded to the identity of the authors (and their institutions) of the manuscripts they are reviewing has been hotly debated for more than 30 years [9]. Knowledge of the authors' identities and their institutional affiliations may affect the fairness of the review process [4, 9]. Identifying authors may be beneficial to well-known authors and detrimental to younger investigators [10]. Reviewers might let inferior papers "slide" if they are submitted from a prestigious researcher or institution [11]. Potential biases of reviewers may relate to the authors, their nationalities, their institutions or institutional status, or their prior manuscript submission history [12]. According to Davidoff [13], "bias is the great demon of science," and, analogous to having musicians audition behind a screen, "we should be doing everything we can to remove the personal elements from the process of making intellectual judgments about biomedical papers."
Initial investigations revealed a variety of advantages in blinding manuscript reviewers to the identities of authors and their institutions. For instance, an analysis of 1051 articles in 28 economics journals (13 journals with double-blinded peer review and 15 with single-blinded review) revealed that articles published in journals with double-blinded review were more likely to be cited in future articles [1]. The results of more recent investigations have been less convincing that blinding reviewers provides any advantages on the basis of statistical analyses [5,6,7, 14], and relatively few biomedical journals have implemented double-blinded peer review policies (19% of 114 English language biomedical journals surveyed in 1988 [12]). Additionally, because the editor ultimately determines the fate of a manuscript, some authorities maintain that blinding or unblinding of a manuscript's authors and institutions to a reviewer does not affect final manuscript disposition [12].
As in our study, several previous smaller series have examined the frequency at which reviewers or editors who examined manuscripts with a blinded title page could determine the identity of the authors of those manuscripts. Justice et al. [7] found that the blinded reviewers unmasked the identities of the authors 32% of the time. In a study by McNutt et al. [3], blinding was unsuccessful mainly because authors cited their own work in 31% of the manuscripts. Only 58% of the 293 blinded reviewers in the study by van Rooyen et al. [5] could not identify the authors of the manuscripts that they reviewed. Of 312 blinded reviewers who examined a manuscript with a blinded title page for the American Journal of Public Health, a 47% incidence of potential unblinding occurred; of these instances, the authors, their institutions, or both were correctly identified by the reviewers 84% of the time [9]. Reviewers could identify the authors or their institutions because of self-referencing by authors (62% of unblinding instances) and and "personal knowledge" of the authors and their work (38% of instances). However, 75% of the reviewers stated that they favored a blinded review process [9].
Davidoff [13] critiqued the recent studies that showed no major advantages of blinding [5,6,7, 14] because blinding was unsuccessful in a large percentage of the manuscripts. He concluded that "the hypothesis that masking does not improve the objectivity and quality of reviewers has therefore not been adequately tested." Masking author identity is more likely to be important for "influential" authors, precisely those whose identities are difficult, if not impossible, to hide. Davidoff also concluded that a "rush to judgment about the non utility of masking seems premature" [13].
Previous studies that have examined the incidence and nature of unblinding by authors of manuscripts submitted to biomedical publications differ from ours in that the reviewers or editors could also guess the authors' identities by personal knowledge. Our study determined only whether authors were honoring the unblinding policy of the two journals as published in their respective "Publication Information for Authors." We agree that it would be impossible to completely blind every reviewer, even if every group of authors adhered to an unblinding policy or every manuscript was blinded by medical journal editorial staff, because many reviewers, for a variety of reasons, will be aware of which researchers are working in specific subspecialty areas [8, 9, 15].
In conclusion, 34% of 880 prospectively inventoried original manuscripts submitted to two radiology journals contained information that potentially or definitely unblinded the identities of the authors, their institutions, or both. Blatant unblinding, such as statement of authors' names or institutions in the body of the text of a manuscript, can be masked by a journal's editorial staff before peer review, but it would be impractical for an editorial office to correct every instance of unblinding. Furthermore, it is impossible to truly blind every manuscript as to the identities of the authors and their institutions because of the knowledge that reviewers have about other investigators and investigations in their areas of subspecialty interest. To minimize bias, editors of journals with double-blinded peer review policies may want to reassess how they implement and enforce these policies. Authors submitting manuscripts to these journals should take steps to avoid unblinding their identities and their institutions.
Acknowledgments
We thank the editorial staffs at Radiology and
RadioGraphics for their assistance in this project, especially
Deborah L. Hogan and Cynthia G. Rogers. Glenn A. Krinsky at New York
University is thanked for the concepts that prompted this study.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
This article has been cited by other articles:
|
|
 |
|
  D. S. Liebeskind, D. S. Katz, A. V. Proto, and W. W. Olmsted The Fallacy of Double-Blinded Peer Review Am. J. Roentgenol., November 1, 2003; 181 (5): 1422 - 1423. [Full Text] [PDF]
|
|
|
|
 |
|
 D. S. Katz, H.-J. Wagner, S. Grampp, T. T. Miller, D. P. Frush, W. C. G. Peh, and K. Applegate The RSNA Editorial Fellowship: Editorial Fellows' Perspective Radiology, February 1, 2003; 226(2): 309 - 311. [Full Text] [PDF]
|
|
|
|
|
|
L. F. Rogers Manuscripts: To Blind, or Not To Blind, That Is the Question Am. J. Roentgenol., December 1, 2002; 179(6): 1373 - 1373. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
