American Journal of Roentgenology, Vol 166, 517-521, Copyright © 1996 by American Roentgen Ray Society

ARTICLES

Simple steps for improving multiple-reader studies in radiology

NA Obuchowski and RC Zepp
Department of Biostatistics and Epidemiology, The Cleveland Clinic Foundation, OH 44195-5196, USA.

Multiple-reader study designs have become popular in the radiology literature. We reviewed the major papers published in the American Journal of Roentgenology in the first 4 months of each of the years 1990 and 1995. The review was restricted to prospective studies of image interpretation. In the 1990 literature, we noted eight multiple- reader and 18 single-reader studies; in contrast, in the 1995 literature, we found 29 multiple-reader and eight single-reader studies. This trend reflects an increased awareness of the importance of multiple-reader studies. We examined the Results sections of the 29 multiple-reader studies from 1995 to assess the authors' motives for incorporating such a design. In 16 studies (55%), readers independently interpreted all images. However, the authors usually reported only the average interpretation of the readers; in only seven of the 29 studies (24%) did the authors describe differences among readers' interpretations. In 13 studies, interpretations were performed exclusively through "consensus reading." The method(s) used to achieve a consensus often were not explained. Only two of the 29 studies had more than three readers. In contrast, all of these studies included multiple patients. The average patient sample size was 45. Furthermore, differences observed among patients were routinely reported and/or depicted.
CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

This article has been cited by other articles:

				R. F. Wagner, C. A. Beam, and S. V. Beiden Reader Variability in Mammography and Its Implications for Expected Utility over the Population of Readers and Cases Med Decis Making, November 1, 2004; 24(6): 561 - 572. [Abstract] [PDF]

				N. A. Obuchowski How Many Observers Are Needed in Clinical Studies of Medical Imaging? Am. J. Roentgenol., April 1, 2004; 182(4): 867 - 869. [Full Text] [PDF]

				H Ohnishi, A Yokoyama, Y Yasuhara, A Watanabe, T Naka, H Hamada, M Abe, K Nishimura, J Higaki, J Ikezoe, et al. Circulating KL-6 levels in patients with drug induced pneumonitis Thorax, October 1, 2003; 58(10): 872 - 875. [Abstract] [Full Text] [PDF]

				S. H. Taplin, C. M. Rutter, J. G. Elmore, D. Seger, D. White, and R. J. Brenner Accuracy of Screening Mammography Using Single Versus Independent Double Interpretation Am. J. Roentgenol., May 1, 2000; 174(5): 1257 - 1262. [Abstract] [Full Text] [PDF]

				B. D. Fletcher, A. S. Glicksman, and P. Gieser Interobserver Variability in the Detection of Cervical-Thoracic Hodgkin's Disease by Computed Tomography J. Clin. Oncol., July 1, 1999; 17(7): 2153 - 2153. [Abstract] [Full Text] [PDF]

				R. Lecesne, P. Taourel, P. M. Bret, M. Atri, and C. Reinhold Acute Pancreatitis: Interobserver Agreement and Correlation of CT and MR Cholangiopancreatography with Outcome Radiology, June 1, 1999; 211(3): 727 - 735. [Abstract] [Full Text]