HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Reiner, B. I. Articles by Hooper, F. J. Search for Related Content PubMed PubMed Citation Articles by Reiner, B. I. Articles by Hooper, F. J. Social Bookmarking                      What's this?

Accuracy of Interpretation of CT Scans: Comparing PACS Monitor Displays and Hard-Copy Images

¹ Department of Radiology, Veterans Affairs Maryland Healthcare System, 10 N. Greene St., Baltimore, MD 21201.
² Department of Radiology, University of Maryland School of Medicine, 22 S. Greene Street, Baltimore, MD 21201.

Received March 18, 2002; accepted after revision June 4, 2002.

 
Address correspondence to B. I. Reiner.

Abstract

Top Abstract Introduction Materials and Methods Results Discussion References

 
OBJECTIVE. The purpose of this study was to determine the relative diagnostic accuracy of radiologists in the interpretation of CT scans using a computer workstation in comparison with using film.

MATERIALS AND METHODS. Four board-certified radiologists with extensive soft-copy experience interpreted 117 CT scans in four anatomic regions using films displayed on an alternator and images displayed on a four-monitor workstation. The radiologists were asked to interpret the scans in their usual fashion and were aware that both the time required to review the study and the accuracy of the reports were being assessed. The radiologists' diagnostic impressions were compared with those of a consensus panel and scored for accuracy.

RESULTS. Soft-copy interpretation using computer workstations was found to produce statistically significant improvement in combined measurements of sensitivity, specificity, and overall accuracy for chest, brain, and chest—abdominal CT scans compared with film interpretation.

CONCLUSION. PACS (picture archiving and communication system) offers radiologists the potential for improved accuracy in CT interpretation compared with traditional film-based interpretation.

Introduction

Top Abstract Introduction Materials and Methods Results Discussion References

 
The use of PACS (picture archiving and communication system) can result in numerous advantages compared with conventional film-based image interpretation. These include increased accessibility of images and reports, better image management with fewer lost and unreviewed scans, and reduced cost [1,2,3,4].

Previous studies evaluated CT interpretation using computer workstations but were limited by long retrieval and display times, a small number and limited variety of scans, reviewer inexperience, limited workstation tools, and unlimited interpretation times [5,6,7,8,9].

We have previously shown an increase of approximately 16% in the speed of radiologists interpreting CT scans using PACS compared with conventional film-based CT interpretation in which radiologists hang their own films on alternators [10]. However, a decrease in the time required to interpret a CT scan is only clinically useful if there is no associated decrease in interpretation accuracy. The purpose of this investigation was to analyze data from the previously published study, which documented a 16% decrease in interpretation times for soft copy in comparison to film and to determine whether a concomitant change in interpretation accuracy occurred.

Materials and Methods

Top Abstract Introduction Materials and Methods Results Discussion References

 
One hundred seventeen randomly selected CT scans were included in the study, closely approximating the proportion of CT examinations performed at our institution. These examinations depicted multiple anatomic regions including the brain (n = 26), chest (n = 35), abdomen and pelvis (n = 33), and chest and abdomen (n = 23). CT was performed on either a PQ 2000 helical scanner (Picker International, Highland Heights, OH) or a nonhelical HiLight Advantage (General Electric Medical Systems, Milwaukee, WI) scanner. Digital images were sent from the scanners to a central PACS server and made available to workstations on the network.

Four board-certified radiologists, experienced with soft-copy interpretation, independently reviewed the CT studies. The CT scans were assigned to each of the four radiologists in such a way that all 117 scans would be interpreted twice, once using a computer workstation and once using film. No individual CT scan was reviewed more than once by an individual radiologist.

Film images were printed with a 12-on-1 format using laser film and the customary window level settings tailored to the anatomic region studied. Hardcopy images (including prior comparison studies when available) were hung on a film alternator by the individual radiologists.

Soft-copy CT interpretations were rendered using tile mode display on a generic PACS workstation with PathSpeed software (General Electric Medical Systems) and Macintosh IIFX (Apple Computers, Cupertino, CA) with four 2048 x 1536 pixel monitors. In all instances, the reviewers were provided with pertinent clinical history and all available prior reports and studies.

Each reviewer was asked to interpret the scans and dictate the results in a customary fashion and then (with the timer turned off) summarize pertinent findings, giving a numeric rank of the clinical significance for each finding, using a standardized data-collection sheet. The ranking of clinical significance was reported on a scale of 1-5: 1 represented an incidental finding of no clinical significance; 2, a finding of uncertain clinical significance; 3, a finding of mild or moderate clinical significance; 4, a finding of high clinical significance; and 5, a medical or surgical emergency. The reviewers were informed that both the amount of time required to interpret the study (not including the summary of pertinent findings and estimation of clinical significance) and diagnostic accuracy in comparison to the findings of a consensus panel would be assessed.

Individual reviewer interpretations were scored against those of a consensus review panel consisting of four radiologists with subspecialty training in neuroradiology and thoracic and abdominal imaging. Both soft- and hard-copy images and the individual interpretations of all of the reviewers were available to the expert review panel. Accuracy was determined by assessing the presence or absence of those radiologic findings believed to be clinically significant—that is, a score of 3 or higher for each anatomic region as deemed by the consensus review panel. Agreement between the individual radiologist and the review panel on both the type and anatomic location of any abnormalities was required. In each individual anatomic region, numerous subregions were defined for individual statistical analysis. For the brain, the subregions included intraaxial, extraaxial, ventricular system, osseous, and posterior fossa regions. For the chest, the subregions analyzed included the heart and great vessels, the pulmonary vasculature, air space, interstitium, mediastinum—hila, pleura, upper abdomen, osseous bone, and soft tissues. For the abdomen, the subregions included lung bases, soft tissues, osseous, gastrointestinal, biliary, genitourinary, lymphatic, and vascular regions.

The primary analysis was to compare the overall accuracy of hard-copy compared to soft-copy interpretation. In summarizing the findings across all anatomic sites and subregions, we compared the results of the four reviewers collectively to the findings of the review panel, which were considered the gold standard. A Z-test was used to compare the proportion of correct interpretations in soft-copy to that of film-based interpretations.

"Sensitivity" is defined as the measure of true-positivity, and "specificity" is defined as the measure of true-negativity. Overall interpretation accuracy was defined as the sum of true-positive and true-negative findings that were greater than the total of all subregions evaluated. The false-positive ratio was defined as the number of false-positive findings that were greater than the sum of false-positive and true-negative findings.

Results

Top Abstract Introduction Materials and Methods Results Discussion References

 
The measurements used to compare CT scan interpretation using PACS and film included sensitivity, specificity, accuracy, and false-positive ratio. In the assessment of chest CT scans, 35 cases were reviewed, with a total of 245 subregions evaluated. Of these 245 sub-regions, 62 (25.3%) were found to have significant abnormal findings, defined as a clinical severity grade of 3 or more. Soft-copy interpretation using PACS was shown to significantly improve sensitivity, specificity, and overall accuracy compared with film interpretation. Both false-positive and false-negative results were greatly reduced using PACS; this reduction showed an overall sensitivity of 0.92, a specificity of 0.98, and an overall accuracy of 0.97. Comparable measures using film included a sensitivity of 0.79, a specificity of 0.88, and an overall accuracy of 0.86. These differences were statistically significant, with a p value of less than 0.001. Of the subregions evaluated, the two categories contributing the greatest degree of improved interpretation accuracy using PACS were the mediastinal—hilar region and airspace. In the individual evaluation of the mediastinal—hilar subregion, 10 additional true-positive results were detected using PACS compared with film. In evaluating air-space disease as an individual subregion, a total of eight false-positive results were reported using film, compared with no false-positive results using PACS.

In the evaluation of brain CT scans, 26 cases were reviewed with 130 subregions evaluated, and 19 abnormal findings (14.6%) of clinical significance were detected. Although the sensitivity at 0.84 (a means to detect true-positivity) was identical comparing PACS and film, improvements with PACS were detected in specificity, overall accuracy, and false-positive ratio. The specificity and overall accuracy using PACS were 0.98 and 0.96, respectively, compared with 0.91 and 0.90 for film (p = 0.08). These differences were due to the reduction of false-positive findings, from 10 using film to two using PACS. The intraaxial subregion accounted for approximately 50% of false-positive findings using film.

In the evaluation of abdominal and pelvic CT scans, no significant differences were observed in comparing soft-copy and hard-copy interpretation. Thirty-three cases were reviewed, with 297 subregions evaluated and 52 clinically significant abnormal findings (17.5%) detected.

When evaluating combined chest and abdominal CT scans, improved sensitivity was observed using PACS compared with film. The sensitivity using PACS was 0.91, whereas the sensitivity using film was 0.75. In these combined chest and abdominal scans (n = 23), 207 subregions were evaluated, with 32 abnormal findings (15.5%) of clinical significance detected. The mediastinum—hila subregion accounted for approximately 40% of this observed sensitivity difference. No significant difference in specificity (detection of true negativity) was observed. A small, statistically insignificant difference was observed in overall accuracy, 0.92 using PACS versus 0.90 using film (p = 0.26).

Discussion

Top Abstract Introduction Materials and Methods Results Discussion References

 
For soft-copy CT interpretation to become an accepted replacement for conventional film-based CT interpretation, accuracy must be clearly established. This need is ironic, given the fact that CT is an inherently digital modality and that the relative accuracy of interpretation using film (compared with using the operator's console or CT workstation) has not been established. However, interpretation of CT images using film has become the de facto standard, and most radiologists have more experience with film than with soft copy, especially when comparing current with previous scans.

Film has a number of inherent advantages: the radiologists' many years of experience with this format; the fact that film alternators and viewboxes have higher luminance than workstations by as much as an order of magnitude; various image-smoothing and other filters typically applied to the images printed on film; and images on film being less subject to monitor variables such as flicker, variable brightness over time, and image degradation over time [11].

The objective of this study was to determine whether a decrease in diagnostic accuracy was associated with the use of soft-copy diagnosis compared with film diagnosis for CT scans. Our results indicate that no compromise in diagnostic accuracy for soft-copy interpretation compared with hard-copy diagnosis was seen. A statistically significant improvement was shown in the overall measure of chest CT soft-copy interpretation accuracy compared with film, with a combined accuracy for soft-copy interpretation of 0.92 versus 0.90 for film interpretation (p = 0.04). This improved diagnostic accuracy using PACS was independent of the individual radiologist, with all individual radiologists showing improved diagnostic accuracy using soft-copy interpretation compared with film. Overall accuracy (agreement with the consensus panel) was relatively high, ranging from 92-97% for soft copy and 86-96% for film. Prior work [12] has shown that in panel studies evaluating interpretation accuracy, an accuracy score greater than 80% is acceptable.

A number of variables should be considered in the evaluation of the accuracy of CT interpretation, including the presence or absence of comparison studies, the specific anatomic region being evaluated, scan complexity, and the mode of image display. Additional consideration should be given to the reviewers themselves, including their level of experience, the amount of time given to interpret the study, closeness of the task to the actual CT reviewing process, and the methodology for determining accuracy. Prior studies [6, 7, 9, 13, 14] have reported mixed results of the accuracy of CT interpretation using computer workstations. These studies were, however, limited in their scope with regard to these variables. For example, interpretation becomes more difficult and time-consuming as the number of images and scan complexity increases [15]. More recent studies have compared interpretation accuracy between conventional film and PACS monitor displays for sonography and conventional radiography and have found no statistically significant differences between soft-copy and hard-copy interpretation [16, 17].

Several theoretic reasons might explain the improved accuracy of CT interpretation using soft-copy workstations. Generic PACS workstations have image review and analysis functions, including preset and variable window level adjustment, zoom and magnification, and pixel—region of interest Hounsfield unit display. Previous work [18] has shown the most commonly used workstation tool in soft-copy interpretation is window level adjustment, a result supported by our experience. Liberal use of customized window level presets allows radiologists to rapidly review multiple window level settings beyond the standard number displayed on hard-copy image displays. Pomerantz et al. [19] found that the routine review of additional window level settings using a computer workstation (beyond those typically printed onto film) resulted in improved conspicuity and characterization of abnormalities in 67% of cases and improvement in clinically important diagnostic information in 18% of cases. They estimated that review of these settings required approximately 40 additional sec ( 7% increase in overall reviewing time). This increase in overall reviewing time, however, has not been substantiated in our most recent experience, with a 16% overall decrease in CT interpretation time using PACS compared with using film [10].

In our population, there was a higher proportion of clinically significant pathologic findings in the chest compared with abdomen—pelvis and brain CT scans. This finding is a reflection of the patients served at our facility, who have disproportionately high levels of pulmonary disease in the form of emphysema and primary—secondary malignancies, which may also have been a contributing factor to the greater differences observed between soft-copy and hard-copy interpretation in the chest compared with abdomen—pelvic CT scans. Liberal use of workstation tools (zoom and scroll, magnification, and window level adjustment) can enhance interpretation by radiologists, particularly when evaluating tightly confined areas with relatively low differences in contrast, such as the mediastinal—hilar region in the chest. In the abdomen, contrast differences are improved, particularly with liberal use of oral and IV contrast material (which is typically used in abdominal—pelvic examinations in our institution). In addition, the presence of intraabdominal and retroperitoneal fat in the abdomen further enhances contrast between adjacent structures and may limit the need for additional workstation tools in soft-copy interpretation. Our experience is that radiologists performing soft-copy interpretation engage in more liberal use of workstation tools for chest CT interpretation than for abdominal—pelvic CT scans. The observed reduction in false-positive findings using soft-copy interpretation of brain CT scans may also be mulifactorial. A smaller number of printed window level settings [2] combined with lack of IV contrast administration may account for similar improvements observed using soft-copy interpretation compared with film. A recent study by Lev et al. [20] reported that detection of brain ischemia was improved with soft-copy interpretation of unenhanced CT scans of the head. This advantage is believed to be due to the liberal use of window level adjustments to accentuate contrast differences between normal and edematous brain tissue.

Other theoretic benefits of soft-copy interpretation could potentially affect interpretation accuracy, and these were not directly assessed in our study design. We did not ask the radiologists to use the "stack" mode [15, 18, 21, 22], in which images are sequentially reviewed in a single window as opposed to the "frame" mode, which we used and which more closely simulates film. Stack mode has been shown to improve both speed and accuracy when reviewed on soft-copy workstations and may have amplified the advantage of soft-copy review in our study. An associated workstation function is the "linked stack" mode, in which two or more separate studies can be linked anatomically and then reviewed in lockstep fashion using multiple cine stacks. This function should have resulted in further increases in review speed for studies with previous scans, but it is not certain whether this feature would have improved diagnostic accuracy. The ready access to previous reports, ordering information, and the patient's electronic medical record may also improve diagnostic accuracy in comparison to a paper-based environment with less convenient access to this information. Future enhancements such as computer-aided diagnosis for lung nodules, image segmentation, texture and tone processing, and online education resources could potentially result in further improvements in diagnostic accuracy.

This study design has a few important limitations. The first is the relatively small number of cases analyzed in the study, making it difficult to establish statistical significance for the individual anatomic regions of the body. For this reason, statistical analysis was performed for the individual subregions in each anatomic region and thereby increased the number of data points.

Second is the manner in which "truth" was established. The expert review panel had access to both hard-copy and soft-copy images along with individual reports of the participating radiologists and follow-up scans. These factors could potentially shift the consensus interpretations toward a larger number of abnormal findings and artificially decrease the diagnostic accuracy. It is, however, unlikely that this situation would result in a significant bias in favor of soft-copy interpretation compared with film-based interpretation. In addition, the classification of findings as "clinically relevant" is somewhat arbitrary. Finally, the fact that participating radiologists were aware of the data collection has further potential to alter the results. However, because both timing and accuracy of interpretation were being evaluated, the potential to overcompensate in either direction would probably be identifiable in the data analysis.

In conclusion, the results of this study suggest that soft-copy interpretation using a computer workstation could improve the accuracy of radiologist interpretation, particularly when radiologists are appropriately trained and have experience in soft-copy interpretation. As workstation hardware and software continue to improve, radiologist productivity and interpretation accuracy should be further enhanced. This combination of improved productivity and diagnostic accuracy provide compelling justification for the continued transition to soft-copy interpretation for CT scans.

Acknowledgments
 
We thank Andrew Dahlke, Steven Pomerantz, and David Rallis, who graciously provided their time and expertise in the interpretation of a large number of the radiologic images included in this study.

References

Top Abstract Introduction Materials and Methods Results Discussion References

 

1. Siegel EL, Diaconis JN, Pomerantz S, Allman R, Briscoe B. Making filmless radiology work. J Digit Imaging 1995;8:151 -155[Medline]
2. Langlotz CP, Even-Shoshan O, Seshadri SS, et al. A methodology for the economic assessment of picture archiving and communication systems. J Digit Imaging 1995;8:95 -102[Medline]
3. Hilsenrath PE, Smith WL, Berbaum KS, Franken EA, Owen DA. Analysis of the cost-effectiveness of PACS. AJR 1991;156:177 -180[Abstract/Free Full Text]
4. Pizer SM, Beard DV. Medical image workstations: state of science and technology. J Digit Imaging 1989;2:185 -193[Medline]
5. Beard DV, Hemminger BM, Denelsbeck KM, Johnston RE. How many screens does a CT workstation need? J Digit Imaging 1994;7:69 -76[Medline]
6. Foley WD, Jacobson DR, Taylor AJ, et al. Display of CT studies on a two-screen electronic workstation versus a film panel alternator: sensitivity and efficiency among radiologists. Radiology 1990;174:769 -773[Abstract/Free Full Text]
7. Berbaum KS, Franken EA, Honda H, McGuire C, Weis RR, Badoon T. Evaluation of a PACS workstation for assessment of body CT studies. J Comput Assist Tomogr 1990;14:853 -858[Medline]
8. Beard DV, Hemminger BM, Perry JR, et al. Interpretation of CT studies: single-screen workstation versus film alternator for fast CT interpretation, Radiology 1993;187:565 -569[Abstract/Free Full Text]
9. Straub WH, Gur D, Good WF, et al. Primary CT diagnosis of abdominal masses in a PACS environment. Radiology 1991;178:739 -741[Abstract/Free Full Text]
10. Reiner BI, Siegel EL, Hooper FJ, Pomerantz S, Dahlke A, Rallis D. Radiologist productivity in the interpretation of CT scans: a comparison of PACS with conventional film images. AJR 2001;176:861 -864[Abstract/Free Full Text]
11. Siegel EL, Reiner BI, Cadogan M. Frequency and impact of high-resolution monitor failure in a filmless imaging department. J Digit Imaging 2000;13:114 -118[Medline]
12. Lowitt MH, Kessler II, Kauffman L, et al. Teledermatology and in person examinations: a comparison of patient and physician perceptions and diagnostic agreement. Arch Dermatol 1998;139:471 -476
13. Hirota H, Shimamoto K, Yamakawa K, et al. Clinical evaluation of newly developed CRT viewing station: CT reading and observer's performance. Comput Med Imaging Graph 1995;19:281 -285[Medline]
14. Beard DV, Hemminger BM, Perry JR, et al. Interpretation of CT studies: single-screen workstation versus film alternator. Radiology 1993;187:565 -569
15. Beard DV, Molina PL, Muller KE, et al. Interpretation time of serial chest CT examinations with stacked-metaphor workstation versus film alternator. Radiology 1995;197:753 -758[Abstract/Free Full Text]
16. Hertzberg BS, Kliewer MA, Paulson EK, et al. PACS in sonography: accuracy of interpretation using film compared with monitor display. AJR 1999;173:1175 -1179[Abstract/Free Full Text]
17. Kundel HL, Polansky M, Dalinka MK, et al. Reliability of soft-copy versus hard-copy interpretation of emergency department radiographs: a prototype study. AJR 2001;177:525 -528[Abstract/Free Full Text]
18. Gay SB, Sobel AH, Young LQ, Dwyer SJ 3rd. Processes involved in reading imaging studies: workflow analysis and implications for workstation development. J Digit Imaging 1997;10:40 -45[Medline]
19. Pomerantz SM, White CS, Krebs TL, et al. Liver and bone window settings for soft-copy interpretation of chest and abdominal CT. AJR 2000;174:311 -314[Abstract/Free Full Text]
20. Lev MH, Farkas J, Gemmete JJ, et al. Acute stroke: improved nonenhanced CT detection—benefits of soft-copy interpretation by using variable window width and center level settings. Radiology 1999;213:150 -155[Abstract/Free Full Text]
21. Mathie AG, Strickland NH. Interpretation of CT scans with PACS image display in stack mode. Radiology 1997;203:207 -209[Abstract/Free Full Text]
22. Memel DS, Berland LL. CT of bowel obstruction: interpretation using cine-paging. (letter) AJR 1995;164:766 -767[Medline]

CiteULike

Complore

Connotea

Del.icio.us

Digg

Reddit

Technorati

This article has been cited by other articles:

				G. P. Krestin, J. C. Miller, S. J. Golding, G. G. Frija, G. M. Glazer, H. G. Ringertz, and J. H. Thrall Reinventing Radiology in a Digital and Molecular Age: Summary of Proceedings of the Sixth Biannual Symposium of the International Society for Strategic Studies in Radiology (IS3R), August 25 27, 2005 Radiology, September 1, 2007; 244(3): 633 - 638. [Full Text] [PDF]

				C. Wittram How I Do It: CT Pulmonary Angiography Am. J. Roentgenol., May 1, 2007; 188(5): 1255 - 1261. [Abstract] [Full Text] [PDF]

				M.-J. Lee, S. Kim, S.-A. Lee, H.-T. Song, Y.-M. Huh, D.-H. Kim, S. H. Han, and J.-S. Suh Overcoming Artifacts from Metallic Orthopedic Implants at High-Field-Strength MR Imaging and Multi-detector CT RadioGraphics, May 1, 2007; 27(3): 791 - 803. [Abstract] [Full Text] [PDF]

				J. M. Wardlaw, A. J. Farrall, D. Perry, R. von Kummer, O. Mielke, T. Moulin, A. Ciccone, M. Hill, and for the Acute Cerebral CT Evaluation of Stroke Stu Factors Influencing the Detection of Early CT Signs of Cerebral Ischemia: An Internet-Based, International Multiobserver Study Stroke, April 1, 2007; 38(4): 1250 - 1256. [Abstract] [Full Text] [PDF]

				M. B. McCarville, H. M. Lederman, V. M. Santana, N. C. Daw, S. J. Shochat, C.-S. Li, and R. A. Kaufman Distinguishing Benign from Malignant Pulmonary Nodules with Helical Chest CT in Children with Malignant Solid Tumors. Radiology, May 1, 2006; 239(2): 514 - 520. [Abstract] [Full Text] [PDF]

				J. E. Roos, B. Chilla, M. Zanetti, M. Schmid, P. Koch, C. W. A. Pfirrmann, and J. Hodler MRI of Meniscal Lesions: Soft-Copy (PACS) and Hard-Copy Evaluation Versus Reviewer Experience. Am. J. Roentgenol., March 1, 2006; 186(3): 786 - 790. [Abstract] [Full Text] [PDF]

				S P Prabhu, S Gandhi, and P R Goddard Ergonomics of digital imaging Br. J. Radiol., July 1, 2005; 78(931): 582 - 586. [Abstract] [Full Text] [PDF]

				M. Scharitzer, M. Prokop, M. Weber, M. Fuchsjager, E. Oschatz, and C. Schaefer-Prokop Detectability of Catheters on Bedside Chest Radiographs: Comparison between Liquid Crystal Display and High-Resolution Cathode-Ray Tube Monitors Radiology, February 1, 2005; 234(2): 611 - 616. [Abstract] [Full Text] [PDF]

				J G Cahir, A H Freeman, and H M Courtney Multislice CT of the abdomen Br. J. Radiol., December 1, 2004; 77(suppl_1): S64 - S73. [Abstract] [Full Text] [PDF]

				M. A. Haider Extending PowerPoint with DICOM Image Support RadioGraphics, November 1, 2003; 23(6): 1683 - 1687. [Abstract] [Full Text] [PDF]

				C. D. Johnson Hard- versus Soft-Copy Interpretation Radiology, June 1, 2003; 227(3): 629 - 630. [Full Text] [PDF]

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Reiner, B. I. Articles by Hooper, F. J. Search for Related Content PubMed PubMed Citation Articles by Reiner, B. I. Articles by Hooper, F. J. Social Bookmarking                      What's this?