| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
PACS in Radiology |
1 Department of Radiology, Veterans Affairs Maryland Healthcare System, 10 N.
Greene St., Baltimore, MD 21201.
2 Department of Radiology, University of Maryland School of Medicine, 22 S.
Greene Street, Baltimore, MD 21201.
Received March 18, 2002;
accepted after revision June 4, 2002.
Address correspondence to B. I. Reiner.
Abstract
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
MATERIALS AND METHODS. Four board-certified radiologists with extensive soft-copy experience interpreted 117 CT scans in four anatomic regions using films displayed on an alternator and images displayed on a four-monitor workstation. The radiologists were asked to interpret the scans in their usual fashion and were aware that both the time required to review the study and the accuracy of the reports were being assessed. The radiologists' diagnostic impressions were compared with those of a consensus panel and scored for accuracy.
RESULTS. Soft-copy interpretation using computer workstations was found to produce statistically significant improvement in combined measurements of sensitivity, specificity, and overall accuracy for chest, brain, and chest—abdominal CT scans compared with film interpretation.
CONCLUSION. PACS (picture archiving and communication system) offers radiologists the potential for improved accuracy in CT interpretation compared with traditional film-based interpretation.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Previous studies evaluated CT interpretation using computer workstations but were limited by long retrieval and display times, a small number and limited variety of scans, reviewer inexperience, limited workstation tools, and unlimited interpretation times [5,6,7,8,9].
We have previously shown an increase of approximately 16% in the speed of radiologists interpreting CT scans using PACS compared with conventional film-based CT interpretation in which radiologists hang their own films on alternators [10]. However, a decrease in the time required to interpret a CT scan is only clinically useful if there is no associated decrease in interpretation accuracy. The purpose of this investigation was to analyze data from the previously published study, which documented a 16% decrease in interpretation times for soft copy in comparison to film and to determine whether a concomitant change in interpretation accuracy occurred.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Four board-certified radiologists, experienced with soft-copy interpretation, independently reviewed the CT studies. The CT scans were assigned to each of the four radiologists in such a way that all 117 scans would be interpreted twice, once using a computer workstation and once using film. No individual CT scan was reviewed more than once by an individual radiologist.
Film images were printed with a 12-on-1 format using laser film and the customary window level settings tailored to the anatomic region studied. Hardcopy images (including prior comparison studies when available) were hung on a film alternator by the individual radiologists.
Soft-copy CT interpretations were rendered using tile mode display on a generic PACS workstation with PathSpeed software (General Electric Medical Systems) and Macintosh IIFX (Apple Computers, Cupertino, CA) with four 2048 x 1536 pixel monitors. In all instances, the reviewers were provided with pertinent clinical history and all available prior reports and studies.
Each reviewer was asked to interpret the scans and dictate the results in a customary fashion and then (with the timer turned off) summarize pertinent findings, giving a numeric rank of the clinical significance for each finding, using a standardized data-collection sheet. The ranking of clinical significance was reported on a scale of 1-5: 1 represented an incidental finding of no clinical significance; 2, a finding of uncertain clinical significance; 3, a finding of mild or moderate clinical significance; 4, a finding of high clinical significance; and 5, a medical or surgical emergency. The reviewers were informed that both the amount of time required to interpret the study (not including the summary of pertinent findings and estimation of clinical significance) and diagnostic accuracy in comparison to the findings of a consensus panel would be assessed.
Individual reviewer interpretations were scored against those of a consensus review panel consisting of four radiologists with subspecialty training in neuroradiology and thoracic and abdominal imaging. Both soft- and hard-copy images and the individual interpretations of all of the reviewers were available to the expert review panel. Accuracy was determined by assessing the presence or absence of those radiologic findings believed to be clinically significant—that is, a score of 3 or higher for each anatomic region as deemed by the consensus review panel. Agreement between the individual radiologist and the review panel on both the type and anatomic location of any abnormalities was required. In each individual anatomic region, numerous subregions were defined for individual statistical analysis. For the brain, the subregions included intraaxial, extraaxial, ventricular system, osseous, and posterior fossa regions. For the chest, the subregions analyzed included the heart and great vessels, the pulmonary vasculature, air space, interstitium, mediastinum—hila, pleura, upper abdomen, osseous bone, and soft tissues. For the abdomen, the subregions included lung bases, soft tissues, osseous, gastrointestinal, biliary, genitourinary, lymphatic, and vascular regions.
The primary analysis was to compare the overall accuracy of hard-copy compared to soft-copy interpretation. In summarizing the findings across all anatomic sites and subregions, we compared the results of the four reviewers collectively to the findings of the review panel, which were considered the gold standard. A Z-test was used to compare the proportion of correct interpretations in soft-copy to that of film-based interpretations.
"Sensitivity" is defined as the measure of true-positivity, and "specificity" is defined as the measure of true-negativity. Overall interpretation accuracy was defined as the sum of true-positive and true-negative findings that were greater than the total of all subregions evaluated. The false-positive ratio was defined as the number of false-positive findings that were greater than the sum of false-positive and true-negative findings.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
In the evaluation of brain CT scans, 26 cases were reviewed with 130 subregions evaluated, and 19 abnormal findings (14.6%) of clinical significance were detected. Although the sensitivity at 0.84 (a means to detect true-positivity) was identical comparing PACS and film, improvements with PACS were detected in specificity, overall accuracy, and false-positive ratio. The specificity and overall accuracy using PACS were 0.98 and 0.96, respectively, compared with 0.91 and 0.90 for film (p = 0.08). These differences were due to the reduction of false-positive findings, from 10 using film to two using PACS. The intraaxial subregion accounted for approximately 50% of false-positive findings using film.
In the evaluation of abdominal and pelvic CT scans, no significant differences were observed in comparing soft-copy and hard-copy interpretation. Thirty-three cases were reviewed, with 297 subregions evaluated and 52 clinically significant abnormal findings (17.5%) detected.
When evaluating combined chest and abdominal CT scans, improved sensitivity was observed using PACS compared with film. The sensitivity using PACS was 0.91, whereas the sensitivity using film was 0.75. In these combined chest and abdominal scans (n = 23), 207 subregions were evaluated, with 32 abnormal findings (15.5%) of clinical significance detected. The mediastinum—hila subregion accounted for approximately 40% of this observed sensitivity difference. No significant difference in specificity (detection of true negativity) was observed. A small, statistically insignificant difference was observed in overall accuracy, 0.92 using PACS versus 0.90 using film (p = 0.26).
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Film has a number of inherent advantages: the radiologists' many years of experience with this format; the fact that film alternators and viewboxes have higher luminance than workstations by as much as an order of magnitude; various image-smoothing and other filters typically applied to the images printed on film; and images on film being less subject to monitor variables such as flicker, variable brightness over time, and image degradation over time [11].
The objective of this study was to determine whether a decrease in diagnostic accuracy was associated with the use of soft-copy diagnosis compared with film diagnosis for CT scans. Our results indicate that no compromise in diagnostic accuracy for soft-copy interpretation compared with hard-copy diagnosis was seen. A statistically significant improvement was shown in the overall measure of chest CT soft-copy interpretation accuracy compared with film, with a combined accuracy for soft-copy interpretation of 0.92 versus 0.90 for film interpretation (p = 0.04). This improved diagnostic accuracy using PACS was independent of the individual radiologist, with all individual radiologists showing improved diagnostic accuracy using soft-copy interpretation compared with film. Overall accuracy (agreement with the consensus panel) was relatively high, ranging from 92-97% for soft copy and 86-96% for film. Prior work [12] has shown that in panel studies evaluating interpretation accuracy, an accuracy score greater than 80% is acceptable.
A number of variables should be considered in the evaluation of the accuracy of CT interpretation, including the presence or absence of comparison studies, the specific anatomic region being evaluated, scan complexity, and the mode of image display. Additional consideration should be given to the reviewers themselves, including their level of experience, the amount of time given to interpret the study, closeness of the task to the actual CT reviewing process, and the methodology for determining accuracy. Prior studies [6, 7, 9, 13, 14] have reported mixed results of the accuracy of CT interpretation using computer workstations. These studies were, however, limited in their scope with regard to these variables. For example, interpretation becomes more difficult and time-consuming as the number of images and scan complexity increases [15]. More recent studies have compared interpretation accuracy between conventional film and PACS monitor displays for sonography and conventional radiography and have found no statistically significant differences between soft-copy and hard-copy interpretation [16, 17].
Several theoretic reasons might explain the improved accuracy of CT
interpretation using soft-copy workstations. Generic PACS workstations have
image review and analysis functions, including preset and variable window
level adjustment, zoom and magnification, and pixel—region of interest
Hounsfield unit display. Previous work
[18] has shown the most
commonly used workstation tool in soft-copy interpretation is window level
adjustment, a result supported by our experience. Liberal use of customized
window level presets allows radiologists to rapidly review multiple window
level settings beyond the standard number displayed on hard-copy image
displays. Pomerantz et al.
[19] found that the routine
review of additional window level settings using a computer workstation
(beyond those typically printed onto film) resulted in improved conspicuity
and characterization of abnormalities in 67% of cases and improvement in
clinically important diagnostic information in 18% of cases. They estimated
that review of these settings required approximately 40 additional sec (
7% increase in overall reviewing time). This increase in overall reviewing
time, however, has not been substantiated in our most recent experience, with
a 16% overall decrease in CT interpretation time using PACS compared with
using film [10].
In our population, there was a higher proportion of clinically significant pathologic findings in the chest compared with abdomen—pelvis and brain CT scans. This finding is a reflection of the patients served at our facility, who have disproportionately high levels of pulmonary disease in the form of emphysema and primary—secondary malignancies, which may also have been a contributing factor to the greater differences observed between soft-copy and hard-copy interpretation in the chest compared with abdomen—pelvic CT scans. Liberal use of workstation tools (zoom and scroll, magnification, and window level adjustment) can enhance interpretation by radiologists, particularly when evaluating tightly confined areas with relatively low differences in contrast, such as the mediastinal—hilar region in the chest. In the abdomen, contrast differences are improved, particularly with liberal use of oral and IV contrast material (which is typically used in abdominal—pelvic examinations in our institution). In addition, the presence of intraabdominal and retroperitoneal fat in the abdomen further enhances contrast between adjacent structures and may limit the need for additional workstation tools in soft-copy interpretation. Our experience is that radiologists performing soft-copy interpretation engage in more liberal use of workstation tools for chest CT interpretation than for abdominal—pelvic CT scans. The observed reduction in false-positive findings using soft-copy interpretation of brain CT scans may also be mulifactorial. A smaller number of printed window level settings [2] combined with lack of IV contrast administration may account for similar improvements observed using soft-copy interpretation compared with film. A recent study by Lev et al. [20] reported that detection of brain ischemia was improved with soft-copy interpretation of unenhanced CT scans of the head. This advantage is believed to be due to the liberal use of window level adjustments to accentuate contrast differences between normal and edematous brain tissue.
Other theoretic benefits of soft-copy interpretation could potentially affect interpretation accuracy, and these were not directly assessed in our study design. We did not ask the radiologists to use the "stack" mode [15, 18, 21, 22], in which images are sequentially reviewed in a single window as opposed to the "frame" mode, which we used and which more closely simulates film. Stack mode has been shown to improve both speed and accuracy when reviewed on soft-copy workstations and may have amplified the advantage of soft-copy review in our study. An associated workstation function is the "linked stack" mode, in which two or more separate studies can be linked anatomically and then reviewed in lockstep fashion using multiple cine stacks. This function should have resulted in further increases in review speed for studies with previous scans, but it is not certain whether this feature would have improved diagnostic accuracy. The ready access to previous reports, ordering information, and the patient's electronic medical record may also improve diagnostic accuracy in comparison to a paper-based environment with less convenient access to this information. Future enhancements such as computer-aided diagnosis for lung nodules, image segmentation, texture and tone processing, and online education resources could potentially result in further improvements in diagnostic accuracy.
This study design has a few important limitations. The first is the relatively small number of cases analyzed in the study, making it difficult to establish statistical significance for the individual anatomic regions of the body. For this reason, statistical analysis was performed for the individual subregions in each anatomic region and thereby increased the number of data points.
Second is the manner in which "truth" was established. The expert review panel had access to both hard-copy and soft-copy images along with individual reports of the participating radiologists and follow-up scans. These factors could potentially shift the consensus interpretations toward a larger number of abnormal findings and artificially decrease the diagnostic accuracy. It is, however, unlikely that this situation would result in a significant bias in favor of soft-copy interpretation compared with film-based interpretation. In addition, the classification of findings as "clinically relevant" is somewhat arbitrary. Finally, the fact that participating radiologists were aware of the data collection has further potential to alter the results. However, because both timing and accuracy of interpretation were being evaluated, the potential to overcompensate in either direction would probably be identifiable in the data analysis.
In conclusion, the results of this study suggest that soft-copy interpretation using a computer workstation could improve the accuracy of radiologist interpretation, particularly when radiologists are appropriately trained and have experience in soft-copy interpretation. As workstation hardware and software continue to improve, radiologist productivity and interpretation accuracy should be further enhanced. This combination of improved productivity and diagnostic accuracy provide compelling justification for the continued transition to soft-copy interpretation for CT scans.
Acknowledgments
We thank Andrew Dahlke, Steven Pomerantz, and David Rallis, who graciously
provided their time and expertise in the interpretation of a large number of
the radiologic images included in this study.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
This article has been cited by other articles:
|
|
 |
|
  P. M. Wong, A. N. Lees, J. Louw, F. Y. Lee, N. French, K. Gain, C. P. Murray, A. Wilson, and D. C. Chambers Emphysema in young adult survivors of moderate-to-severe bronchopulmonary dysplasia Eur. Respir. J., August 1, 2008; 32(2): 321 - 328. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 G. P. Krestin, J. C. Miller, S. J. Golding, G. G. Frija, G. M. Glazer, H. G. Ringertz, and J. H. Thrall Reinventing Radiology in a Digital and Molecular Age: Summary of Proceedings of the Sixth Biannual Symposium of the International Society for Strategic Studies in Radiology (IS3R), August 25 27, 2005 Radiology, September 1, 2007; 244(3): 633 - 638. [Full Text] [PDF]
|
|
|
|
 |
|
 C. Wittram How I Do It: CT Pulmonary Angiography Am. J. Roentgenol., May 1, 2007; 188(5): 1255 - 1261. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M.-J. Lee, S. Kim, S.-A. Lee, H.-T. Song, Y.-M. Huh, D.-H. Kim, S. H. Han, and J.-S. Suh Overcoming Artifacts from Metallic Orthopedic Implants at High-Field-Strength MR Imaging and Multi-detector CT RadioGraphics, May 1, 2007; 27(3): 791 - 803. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. M. Wardlaw, A. J. Farrall, D. Perry, R. von Kummer, O. Mielke, T. Moulin, A. Ciccone, M. Hill, and for the Acute Cerebral CT Evaluation of Stroke Stu Factors Influencing the Detection of Early CT Signs of Cerebral Ischemia: An Internet-Based, International Multiobserver Study Stroke, April 1, 2007; 38(4): 1250 - 1256. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. B. McCarville, H. M. Lederman, V. M. Santana, N. C. Daw, S. J. Shochat, C.-S. Li, and R. A. Kaufman Distinguishing Benign from Malignant Pulmonary Nodules with Helical Chest CT in Children with Malignant Solid Tumors. Radiology, May 1, 2006; 239(2): 514 - 520. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. E. Roos, B. Chilla, M. Zanetti, M. Schmid, P. Koch, C. W. A. Pfirrmann, and J. Hodler MRI of Meniscal Lesions: Soft-Copy (PACS) and Hard-Copy Evaluation Versus Reviewer Experience. Am. J. Roentgenol., March 1, 2006; 186(3): 786 - 790. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S P Prabhu, S Gandhi, and P R Goddard Ergonomics of digital imaging Br. J. Radiol., July 1, 2005; 78(931): 582 - 586. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Scharitzer, M. Prokop, M. Weber, M. Fuchsjager, E. Oschatz, and C. Schaefer-Prokop Detectability of Catheters on Bedside Chest Radiographs: Comparison between Liquid Crystal Display and High-Resolution Cathode-Ray Tube Monitors Radiology, February 1, 2005; 234(2): 611 - 616. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J G Cahir, A H Freeman, and H M Courtney Multislice CT of the abdomen Br. J. Radiol., December 1, 2004; 77(suppl_1): S64 - S73. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. A. Haider Extending PowerPoint with DICOM Image Support RadioGraphics, November 1, 2003; 23(6): 1683 - 1687. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
C. D. Johnson Hard- versus Soft-Copy Interpretation Radiology, June 1, 2003; 227(3): 629 - 630. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
