AJR ARRS Membership
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
 QUICK SEARCH:   [advanced]


     


This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Reiner, B. I.
Right arrow Articles by Rallis, D.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Reiner, B. I.
Right arrow Articles by Rallis, D.
Social Bookmarking
 Add to CiteULike   Add to Complore   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Reddit   Add to Technorati  
What's this?
AJR 2001; 176:861-864
© American Roentgen Ray Society


Radiologists' Productivity in the Interpretation of CT Scans

A Comparison of PACS with Conventional Film

Bruce I. Reiner1,2,3, Eliot L. Siegel1,2, Frank J. Hooper1, Stephen Pomerantz1, Andrew Dahlke1 and David Rallis1

1 Department of Radiology, Veterans Affairs Maryland Health Care System, 10 N. Greene St., Baltimore, MD 21201.
2 University of Maryland School of Medicine, 22 S. Greene St., Baltimore, MD 21201.
3 Department of Radiology, Nanticoke Memorial Hospital, 801 Middleford Rd., Seaford, DE 19973.

Received May 2, 2000; accepted after revision October 6, 2000.

 
Address correspondence to B. I. Reiner, 21779 Cove Ln., Leonardtown, MD 20650.


Abstract
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
OBJECTIVE. We compared radiologists' times in the interpretation of CT using hardcopy films with the interpretation using a soft-copy picture archiving and communication system (PACS) computer workstation.

MATERIALS AND METHODS. One hundred CT examinations were selected at random and reviewed by four board-certified radiologists experienced in soft-copy interpretation. We performed time-motion analysis to determine the total time required to display, interpret, and dictate the individual findings of CT using conventional hard-copy interpretation on a viewbox and soft-copy interpretation, using a four-monitor high-resolution (2048 x 1536 pixel) workstation.

RESULTS. Time-motion analysis showed a reduction of 16.2% in the overall time required for soft-copy interpretation of CT compared with that of film. Time savings with soft-copy interpretation were observed for all four participating radiologists. The benefit of soft-copy interpretation was increased for examinations in which there were comparison studies.

CONCLUSION. We found that soft-copy interpretation of CT using a PACS workstation requires less time than interpretation using conventional film hung on a viewbox. The transition to filmless imaging has the potential to improve radiologists' productivity and report-turnaround time.


Introduction
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
The technologic imperative that has been the driving force advancing radiology over the past 20 years has produced new approaches to the acquisition of medical images and placed radiology at the leading edge of the computer-technology era of modern medicine. Several factors are responsible for this change, including a desire to be on the cutting edge, advantages of marketing, opportunities for financial gain, and protection of specialty turf [1].

As health care costs continue to rise, a requirement exists for quantitative accountability: data to justify the cost-efficacy of these new technologies [2]. Results of this imperative to cost-justify the transition to a picture archiving and communication system (PACS) can take the form of increased revenues, improved operational efficiency, or enhanced staff productivity [3,4,5]. Whereas technologists' productivity has been shown to improve in a filmless imaging environment, little to date has been published quantifying the effect of filmless imaging on radiologists' productivity.

CT is particularly well suited to soft-copy interpretation for two reasons: the data are fundamentally digital and the images are small enough to completely fit on available monitors [6]. Despite the theoretic advantages of soft-copy interpretation of CT, other studies [7,8,9,10,11] have found unacceptably slow interpretation times when compared with those of conventional hard-copy interpretation with film. Our study was undertaken to determine if radiologists' productivity could be enhanced in the routine interpretation of CT with current PACS technology.


Materials and Methods
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
One hundred randomly selected CT scans were included in the study. CT was performed with either a PQ-2000 helical scanner (Marconi Medical Systems, Cleveland, OH) or a HiLight Advantage scanner (General Electric Medical Systems, Milwaukee, WI). Both CT scanners used Digital Image COmmunication in Medicine (DICOM) to transfer images to a commercial PACS (General Electric Medical Systems). Because of the "central" architecture of the system, images are stored in a redundant array of inexpensive disks (RAID) server with no local storage at the workstations and are retrieved on demand rather than being routed to the local workstation hard drive.

Four board-certified radiologists, each experienced with PACS, individually reviewed all CT scans. The scans were divided anatomically, including brain (n = 14), chest (n = 53), and abdomen and pelvis (n = 51), in a proportion representative of the typical distribution of CT scans in our radiology department. Interpretations were made with both conventional film-based images displayed on a CT alternator and soft-copy images displayed on a four-monitor 2048 x 1536 pixel Macintosh II workstation (Apple Computers, Cupertino, CA). Radiologists were aware of the nature of the study and were asked to interpret the CT scans in their usual and customary fashion.

Hard-copy films were hung on a CT alternator by the radiologists, who included prior comparison studies when available. Previous reports were available in the patient's film folder. Because our institution does not regularly print film images, all relevant current and previous scans used for the study were manually printed and placed in a standard film jacket. We printed film images in a standard 12-on-one format using laser film, with the number of window and level settings used before the transition to filmless operation based on the anatomic region studied.

We displayed soft-copy images on the four-monitor workstation, using frame-display mode, in which images are displayed in a horizontal fashion from left to right, similar to the manner of image display used in hard-copy film. The number of individual images displayed on each monitor screen (format) was individually selected by the interpreting radiologist and typically consisted of nine or 12 images. Each radiologist used window or level presets that were preprogrammed into the computer workstation. Comparison images and reports were available at the workstations. The image-retrieval speed averaged approximately five CT scans per second. Time measurements were recorded by an independent observer, who recorded the total time for CT interpretation, beginning with the selection of the patient's folder and ending with completion of dictation. The individual processes of image display, review, interpretation, and report dictation were grouped together because most radiologists do not perform these tasks independently and sequentially. Rather, image review and interpretation began as the first images began to display and often continued until the end of dictation.

Statistical analysis consisted of both one-way and two-way analyses of variance, which were used to evaluate the time differences between conventional hard-copy interpretation with film and soft-copy interpretation with PACS. Data were evaluated collectively and individually, according to anatomic region and the interpreting radiologist.


Results
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
There was an overall reduction of 16.2% in the total time required for CT interpretation with soft-copy interpretation compared with that for conventional hard-copy film. All four participating radiologists revealed time savings in CT interpretation using PACS, compared with film, with a range of 0.10-2.20 min (Table 1).


View this table:
[in this window]
[in a new window]
 
TABLE 1 Average Time of CT Interpretation: Soft Copy Versus Film

 

Table 2 summarizes time savings with PACS as a function of anatomic region, with and without the presence of comparison studies. In the absence of comparison studies, interpretation of both chest and abdominal or pelvic CT scans revealed substantial time savings with soft-copy interpretation. Interpretation of CT scans of the brain, however, did not show time savings with PACS, presumably because of the limited number of total images and window or level settings typically encountered.


View this table:
[in this window]
[in a new window]
 
TABLE 2 Average Time of CT Interpretation by Anatomic Region: Soft-Copy Versus Film

 

The presence of a comparison study produced significantly greater time savings with PACS for each individual anatomic region studied. Using two-way analysis of variance, we found a statistically significant difference in evaluating the two-way interaction between the presence or absence of comparison studies and PACS versus film (p < 0.001). These time differences were found to be greatest as the scan complexity and number of images increased. The greatest mean time savings observed was with chest CT interpretations, which revealed a time savings with PACS of 1.79 min without a comparison study and 4.44 min with a comparison study. Whereas brain CT-interpretation times for film and PACS were identical without comparison studies, the presence of comparison studies did result in a mean time savings of 1.53 min with PACS.


Discussion
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
The process of CT imaging involves three fundamental steps: data acquisition, image display, and image interpretation [12]. Prior work has revealed a 60% reduction in the time required for a technologist to perform CT with PACS, compared with the time required for performing CT with the traditional film-based mode [3]. This dramatic reduction in data-acquisition time with PACS is due to the elimination of a number of workflow steps related to film processing, resulting in the potential to significantly increase technologists' productivity with filmless CT.

We are not aware of any published data concerning the relative number of radiologists who hang their own films on a viewbox or film alternator compared with those who have them prehung by file room staff. Anecdotal discussions with colleagues suggest that radiologists hang their own CT films in most facilities. Even when CT scans are prehung by file room staff, radiologists often spend time rearranging film order and retrieving additional comparison studies and reports from the patient's film folder. This additional time ultimately diminishes the theoretic timesaving advantage of prehung films [13].

Whereas time measurements for data retrieval are fairly straightforward, separating the processes of image review and interpretation is more difficult. The processes of image review, interpretation, and dictation tend to overlap, with radiologists often reviewing images as they are dictating. This system makes it impractical to determine the relative contribution to time savings of soft-copy-scan display, review, and interpretation or dictation.

Studies evaluating comparative CT-interpretation times with hard-copy and soft-copy displays have reported discrepant results. Two studies [11, 14] have reported CT-interpretation time with computer workstations to be comparable to conventional-film viewbox interpretation, whereas other studies [7, 9] have reported prolongation of CT-interpretation time with computer workstations. These prior studies, however, were limited by a number of factors, including the number of monitors, speed and functionality of the computer workstations, extent of anatomy and pathology evaluated, and lack of inclusion of comparison studies and reports. The relatively poor performance of the computer workstations in these prior studies was probably related to time delays in the retrieval and display of multiple CT scans and in the amount of time required to achieve the desired display of the images before interpretation.

Our study attempts to address some limitations of previous reports and takes advantage of improved software, optimizing hanging protocols, use of window or level presets, greater radiologist experience using PACS, and faster image retrieval rates. In evaluating user requirements for PACS, Gur et al. [15] reported that observers consider image display slow if the time to display a full screen exceeds a few seconds. Image retrieval and display times in previous studies varied from 5 to 20 sec per monitor [7], depending on the format and technology being used. With the workstations used in this study and 9:1 and 12:1 image formatting, a full screen was displayed in 1-2 sec, significantly better than the performance in those previous studies. Using a four-monitor workstation allows 36-48 images to be quickly displayed and available to the radiologist in less time than would be required to manually hang CT films on a viewbox or alternator for film interpretation. Additionally, radiologists can begin the process of image review and interpretation as soon as the first few images are displayed rather than waiting for all the images to be displayed on all available monitors. The process of image review as the images are being hung is more difficult in a film-based environment in which the radiologists typically place a number of films on the viewbox before the image-review process begins.

Soft-copy interpretation with PACS also offers an alternative to the traditional frame-mode display used in this study. This alternative image-display format is referred to as stack mode, in which CT images are effectively stacked on top of one another on the display monitor. The stack of images can be scrolled back and forth at the direction of the reviewer, using the mouse as a control. Currently available refresh rates can be as high as 30 images per second [12] or greater, allowing the observer to review the scan in a cine fashion. This method has the theoretic advantage of improving interpretation of sequential changes in images but has not yet been used to full potential for a number of reasons. First, not all PACS vendors have stack-mode or cine capabilities fully incorporated into their workstations. A study by Ghosh et al. [16] described a software program developed to merge both frame and stack modes of operation and to allow the radiologist to instantaneously switch between the two different display modes. Newer PACS workstations have been upgraded with software that allows stack-mode display to fully complement frame-mode display capabilities.

Another perceived limitation of stack mode is one of familiarity of the radiologist. Most radiologists (including those participating in this study) were trained in a film-based environment in which hard-copy images were displayed in a tile format. Having this built-in preference, many radiologists prefer to maintain similar styles of interpretation when transitioning from film to filmless operation.

To date, only a few published studies have reported the use of stack-mode display in CT interpretation [11, 12, 17, 18]. Although use of stack mode has theoretic advantages, there have been limited practical experience and published data to date verifying its superiority over the more traditional frame-mode display for CT interpretation. More scientific investigation is required to determine the effect using stack mode would have on radiologists' productivity.

The image retrieval and display time for tile-mode display is about five images per second for the central PACS architecture used in this study, in which images are fetched from a RAID server. This would allow a 48-image scan displayed on a 12:1 format in approximately 10 sec. Every PACS system differs in its relative performance; this difference in turn affects radiologists' efficiency. Generally, the faster the image is retrieved, the greater the potential is to enhance radiologists' productivity with soft-copy interpretation. An important factor in image retrieval with PACS is the software that automates the processes and the arrangement for current and historic scans. Intelligent navigation and image-arrangement software are critical in optimizing radiologists' productivity in CT interpretation with PACS.

Gay et al. [19] have shown adjustment of window or level settings to be the most commonly used workstation tool in soft-copy interpretation; their findings have been supported by our experience. Using computer presets, we found that multiple window- or level-setting combinations (lung, mediastinum, or bone) can be invoked in a few seconds. This feature results in faster image display over a wider range of window or level settings when compared with conventional film interpretation, which requires time-intensive hanging and rearranging multiple sheets of film of the same anatomy printed in various window or level settings. With PACS soft-copy interpretation, radiologists have the additional benefit of electronically linking images from two different studies, allowing greater ease of comparison.

As additional hardware and software further evolve, enhancements in radiologists' productivity with soft-copy interpretation should be realized. One example of improved workflow through software refinements was described in a recent article by Gale et al. [13], who reported development of software designed for automated image display, consolidating multiple commands into a single mouse click. The net result is more efficient retrieval and display of sequential studies, resulting in a measured decrease in the interpretation time of chest radiographs. Other authors have described similar advances in PACS interface design to enhance radiologists' workflow with soft-copy interpretation [20,21,22].

Additional hardware advances in monitor technology to improve radiologists' productivity include brighter high-resolution monitors; faster refresh rates to minimize flicker; and greater consistency of brightness, contrast, and tube color to help reduce radiologists' fatigue. These proposed modifications will undoubtedly lead to further enhancements in the productivity of radiologists.

In conclusion, time measurements comparing film-based and filmless CT interpretations reveal substantial time savings with PACS. These improvements were found to be independent of the radiologist and scan type and were accentuated as the scan complexity and number of images increased. As hardware and software developments continue to enhance PACS workstation technology, the productivity gains realized by radiologists in a soft-copy—reviewing environment should continue to increase.


References
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 

  1. Hillman BJ. New imaging technology and cost containment. AJR 1994;162-503 -506[Abstract/Free Full Text]
  2. Hendee WR, Brown EF, Stanley RJ, Thrall JH, Zylak CJ. Colliding forces in radiology: technologic imperatives, resource limitations, and accountability demands. RadioGraphics 1994;14:647 -653[Abstract]
  3. Reiner BI, Siegel EL, Hooper FJ, Glasser D. Effect of film-based versus filmless operation on the productivity of CT technologist. Radiology 1998;207:481 -485[Abstract/Free Full Text]
  4. Reiner BI, Siegel EL, Flagle C, Hooper FJ, Cox RE, Scanlon M. Effect of filmless imaging on the utilization of radiology services. Radiology 2000;215:163 -167[Abstract/Free Full Text]
  5. Siegel EL, Diaconis JN, Pomerantz S, Allman R, Briscoe B. Making filmless radiology work. J Digit Imaging 1995;8:151 -155[Medline]
  6. Beard DV, Hemminger BM, Demelsbeck KM, Johnston RE. How many screens does a CT workstation need? J Digit Imaging 1994;7:69 -76[Medline]
  7. Hirota H, Shimamoto K, Yamakawa K, et al. Clinical evaluation of newly developed CRT viewing station: CT reading and observer's performance. Comput Med Imaging Graph 1995;19:281 -285[Medline]
  8. Kato H, Kubota G, Kojima K, et al. Preliminary time-flow study: comparison of interpretation times between PACS workstations and film. Comput Med Imaging Graph 1995;19:261 -265[Medline]
  9. Foley WD, Jacobson DR, Taylor AJ, et al. Display of CT studies on a two-screen electronic workstation versus a film panel alternator: sensitivity and efficiency among radiologists. Radiology 1990;174:769 -773[Abstract/Free Full Text]
  10. Berbaum KS, Franken EA, Honda H, McGuire C, Weiss RR, Barloon T. Evaluation of a PACS workstation for assessment of body CT studies. J Comput Assist Tomogr 1990;4:853 -858
  11. Straub WH, Gur D, Good WF, et al. Primary CT diagnosis of abdominal masses in a PACS environment. Radiology 1991;178:739 -743[Abstract/Free Full Text]
  12. Mathie AG, Strickland NH. Interpretation of CT scans with PACS image display in stack mode. Radiology 1997;203:207 -209[Abstract/Free Full Text]
  13. Gale DR, Gale ME, Schwartz RK, Muse VV, Walker RE. An automated PACS workstation interface: a timesaving enhancement. AJR 2000;174:33 -36[Abstract/Free Full Text]
  14. Beard DV, Hemminger BM, Perry JR, et al. Interpretation of CT studies: single-screen workstation versus film alternator. Radiology 1993;187:565 -569[Abstract/Free Full Text]
  15. Gur D, Fuhrman CR, Thaete FL. Requirements for PACS: users' perspectives. RadioGraphics 1993;3:457 -460
  16. Ghosh S, Andriole KP, Avrin DE. ViewPort: an object-oriented approach to integrate workstation software for tile and stack mode display. J Digit Imaging 1997;10:177 -179[Medline]
  17. Beard DV, Pisano ED, Denelsbeck KM, et al. Eye movement during CT interpretation: eye-tracker results and image display-time implications. J Digit Imaging 1994;7:189 -192[Medline]
  18. Memel DS, Berland LL. CT of bowel obstruction: interpretation using cine-paging. (letter) AJR 1995;164:766 -767[Medline]
  19. Gay SB, Sobel AH, Young LQ, Dwyer SJ III. Processes involved in reading imaging studies: workflow analysis and implications for workstation development. J Digit Imaging 1997;10:40 -45[Medline]
  20. Beard D. Designing a radiology workstation: a focus on navigation during the interpretation task. J Digit Imaging 1990;3:152 -163[Medline]
  21. Strickland NH, Allison DJ. Default display arrangements of images on PACS monitors. Br J Radiol 1995;68:252 -260[Abstract]
  22. Dreyer KJ, Mehta A, Johnson KM, Schultz T, Sack D. The primary interpretation workstation: information beyond image data. J Digit Imaging 1998;11:16 -19[Medline]

Add to CiteULike CiteULike   Add to Complore Complore   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Reddit Reddit   Add to Technorati Technorati    What's this?


This article has been cited by other articles:


Home page
RadiologyHome page
G. P. Krestin, J. C. Miller, S. J. Golding, G. G. Frija, G. M. Glazer, H. G. Ringertz, and J. H. Thrall
Reinventing Radiology in a Digital and Molecular Age: Summary of Proceedings of the Sixth Biannual Symposium of the International Society for Strategic Studies in Radiology (IS3R), August 25 27, 2005
Radiology, September 1, 2007; 244(3): 633 - 638.
[Full Text] [PDF]


Home page
RadiologyHome page
J. H. Thrall
Reinventing Radiology in the Digital Age: Part I. The All-Digital Department
Radiology, August 1, 2005; 236(2): 382 - 385.
[Full Text] [PDF]


Home page
Br. J. Radiol.Home page
S P Prabhu, S Gandhi, and P R Goddard
Ergonomics of digital imaging
Br. J. Radiol., July 1, 2005; 78(931): 582 - 586.
[Abstract] [Full Text] [PDF]


Home page
Br. J. Radiol.Home page
J G Cahir, A H Freeman, and H M Courtney
Multislice CT of the abdomen
Br. J. Radiol., December 1, 2004; 77(suppl_1): S64 - S73.
[Abstract] [Full Text] [PDF]


Home page
Jpn J Clin OncolHome page
H. Furukawa, M. Endo, T. Aramaki, N. Morimoto, T. Uematsu, S. Yukizawa, and S. Yuen
Picture Archiving and Communication System Introduced to a New Japanese Cancer Center Hospital
Jpn. J. Clin. Oncol., July 1, 2004; 34(7): 425 - 428.
[Abstract] [Full Text] [PDF]


Home page
RadioGraphicsHome page
M. G. Harisinghani, M. A. Blake, M. Saksena, P. F. Hahn, D. Gervais, M. Zalis, L. da Silva Dias Fernandes, and P. R. Mueller
Importance and Effects of Altered Workplace Ergonomics in Modern Radiology Suites
RadioGraphics, March 1, 2004; 24(2): 615 - 627.
[Abstract] [Full Text] [PDF]


Home page
RadiologyHome page
C. D. Johnson
Hard- versus Soft-Copy Interpretation
Radiology, June 1, 2003; 227(3): 629 - 630.
[Full Text] [PDF]


Home page
Am. J. Roentgenol.Home page
B. I. Reiner, E. L. Siegel, and F. J. Hooper
Accuracy of Interpretation of CT Scans: Comparing PACS Monitor Displays and Hard-Copy Images
Am. J. Roentgenol., December 1, 2002; 179(6): 1407 - 1410.
[Abstract] [Full Text] [PDF]


Home page
RadiologyHome page
S. C. Wagner, W. B. Morrison, J. A. Carrino, M. E. Schweitzer, and H. Nothnagel
Picture Archiving and Communication System: Effect on Reporting of Incidental Findings
Radiology, November 1, 2002; 225(2): 500 - 505.
[Abstract] [Full Text] [PDF]


Home page
Am. J. Roentgenol.Home page
M. Bhargavan, J. H. Sunshine, and B. Schepps
Too Few Radiologists?
Am. J. Roentgenol., May 1, 2002; 178(5): 1075 - 1082.
[Abstract] [Full Text] [PDF]


This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Reiner, B. I.
Right arrow Articles by Rallis, D.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Reiner, B. I.
Right arrow Articles by Rallis, D.
Social Bookmarking
 Add to CiteULike   Add to Complore   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Reddit   Add to Technorati  
What's this?


HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS