AJR ARRS: Your Link to CME
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
 QUICK SEARCH:   [advanced]


     


This Article
Right arrow Abstract Freely available
Right arrow Figures Only
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Li, F.
Right arrow Articles by MacMahon, H.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Li, F.
Right arrow Articles by MacMahon, H.
Social Bookmarking
 Add to CiteULike   Add to Complore   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Reddit   Add to Technorati  
What's this?
Hotlight (NEW!)
Right arrow
What's Hotlight?
DOI:10.2214/AJR.07.2875
AJR 2008; 190:886-891
© American Roentgen Ray Society


Original Research

Improved Detection of Small Lung Cancers with Dual-Energy Subtraction Chest Radiography

Feng Li1, Roger Engelmann1, Kunio Doi1 and Heber MacMahon1

1 All authors: Kurt Rossmann Laboratories for Radiologic Image Research and the Department of Radiology, The University of Chicago, 5841 S Maryland Ave., MC-2026, Chicago, IL 60637.

Received July 16, 2007; accepted after revision October 9, 2007.

 
This work was supported in part by USPHS grants CA61625, CA09119, and EB00341 and by a research grant from Riverain Medical.

CAD technologies developed in the Kurt Rossmann Laboratories for Radiologic Image Research have been licensed to R2 Technology, Deus Technologies, Riverain Medical, Mitsubishi Space Software Co., Median Technologies, GE Healthcare Corporation, and Toshiba Corporation.

K. Doi and H. MacMahon are shareholders of R2 Technology, Sunnyvale, CA, and are consultants for Riverain Medical, Dayton, OH.

Presented at the 2006 annual meeting of the Radiological Society of North America, Chicago, IL.

Address correspondence to F. Li (feng{at}uchicago.edu).


Abstract
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
OBJECTIVE. The objective of our study was to retrospectively evaluate whether the use of dual-energy subtraction chest radiographs can improve radiologists' performance for the detection of small previously missed lung cancers.

MATERIALS AND METHODS. Dual-energy subtraction chest radiographs of 19 patients with previously missed nodular cancers, in which the radiology report did not mention a nodule that was visible in retrospect, were selected. Dual-energy subtraction radiographs of 19 patients with cancer and 16 patients without cancer were used for an observer study. Six radiologists indicated their confidence level regarding the presence of a lung cancer and, if they thought a cancer was present, also marked the most likely position for each lung, first using standard posteroanterior and lateral chest radiographs and then using both soft-tissue and bone dual-energy subtraction images along with standard radiographs. Receiver operating characteristic (ROC) curves were used to evaluate the observers' performance. The indicated locations of cancers and false-positives were also analyzed.

RESULTS. The average area under the ROC curve (Az) value for the six radiologists was improved from 0.718 to 0.816, a statistically significant amount (p = 0.004), and the average sensitivity (correct localizations) for 19 previously missed cancers was also significantly improved from 40% to 59% (p = 0.008) with the aid of dual-energy subtraction images. The average number of false-positive (incorrect) localizations on 70 lungs was 10 without and nine with dual-energy subtraction images (p = 0.785).

CONCLUSION. Dual-energy subtraction chest radiography has the potential to improve radiologists' performance for the detection of small missed lung cancers.

Keywords: diagnostic radiology • digital radiography • dual-energy subtraction radiography • lung neoplasms • observer performance study


Introduction
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
With the evolution from film-based to digital systems, there have been many remarkable advances in projection chest imaging [1, 2]. Dual-energy subtraction techniques can distinguish bone from soft tissue [3, 4], and dual-energy subtraction chest radiography has been shown by several observer performance studies to improve the observers' ability to detect and characterize lung nodules [57].

Dual-energy subtraction chest radiography has been used routinely for more than 10 years in our institution. We recently studied lung cancers previously missed by radiologists on chest radiographs in 34 patients [8], over half of whom also had dual-energy subtraction images. We noted that many of these lung cancers were very subtle on the standard radiographs but were relatively obvious on the soft-tissue images. We suspect that the radiologists did not review the dual-energy subtraction images in many such cases either because they were unaware that dual-energy subtraction images were available or because the dual-energy subtraction images were not easily available for viewing due to technical issues at that time. The purpose of our observer study was to evaluate whether dual-energy subtraction chest radiographs can improve radiologists' performance for the detection of small lung cancers.


Materials and Methods
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
Institutional review board approval was obtained, and the requirement for informed patient consent was waived. Our study was compliant with HIPAA.

Patient Database and Reference Standard
The records of the cancer registry at the University of Chicago Hospitals were reviewed to identify all patients diagnosed with lung cancer from January 2001 to November 2004 (n = 821). All patients with chest radiographs on file before treatment were reviewed, and the relevant radiographs and reports were analyzed (n = 314). From the 314 patients, 34 standard posteroanterior (PA) digital radiographs of 34 patients were identified in which a nodular cancer was present on the standard PA image in retrospect, but had not been mentioned in the initial report. The location of each missed cancer on the 34 standard PA chest radiographs was identified by consensus of two radiologists, with all confirmed by CT. All of the cancers were confirmed using biopsy or surgery as the reference standard. The two radiologists had 15 and 30 years of experience, respectively, in interpreting chest radiographs. Analysis of the 34 missed lung cancers on 34 standard PA chest radiographs has been reported previously in a study of computer-aided detection (CAD) [8].

Dual-Energy Subtraction Chest Radiographs for Observer Study
Among the 34 patients with missed lung cancers, 21 had dual-energy subtraction radiographs, of which 19 were available for review. All of these missed cancers were visible in retrospect on the standard PA images by consensus of the two radiologists, and all were visible on the dual-energy subtraction images as well. Among these 19 patients, three had two primary cancers in different lungs. One patient had two lung cancers, both of which were identifiable on the soft-tissue image, but only one of which was identifiable on the standard image. Two patients had a small nodular cancer (< 20 mm) that had been mentioned in the report in addition to the missed cancer. These two detected cancers and the one that was identifiable only on the dual-energy subtraction image were excluded from the analysis of observer results, although the images were included in the observer test.

In total, 19 patients with missed lung cancers on standard (Fig. 1) or dual-energy subtraction radiographs (or both) were used in our observer study. The 19 patients with missed cancers included 12 men and seven women with a mean age of 69 years (range, 58–87 years). The mean interval between CT and corresponding dual-energy subtraction radiography for the 19 patients was 11 months (five within 1 month, six at 2–12 months, five at 13–24 months, and three at 25–32 months). Eight lung cancers were located in the right lung (four in the right upper lobe, one in the right middle lobe, and three in the right lower lobe), and 11 were located in the left lung (seven in the left upper lobe and four in the left lower lobe). The pathologic diagnoses for the 19 cancers included 17 non–small cell carcinomas (including seven adenocarcinomas, seven squamous cell carcinomas, and three unspecified non–small cell carcinomas) and two small cell carcinomas.


Figure 1
View larger version (155K):
[in this window]
[in a new window]
[as a PowerPoint slide]
 
Fig. 1 Locations of 19 missed cancers (eight in right lungs and 11 in left lungs) in 19 patients are shown superimposed on posteroanterior (PA) chest radiograph.

 

Before the observer study, the 19 missed cancers were graded for subtlety (from extremely subtle to extremely obvious on a 1–10 scale) on the standard images by the same two radiologists independently, and their numeric ratings were averaged. The cases consisted of 12 with ratings of 1 to 3 (extremely to very subtle) and seven with ratings of 3.5 to 6.5 (subtle to relatively obvious). The mean diameter (average of the length and width) of the 19 identifiable lung cancers was 15.6 mm (range, 7–22 mm) on standard images as measured by one radiologist.

Sixteen control subjects (mean age, 66 years; range, 50–79 years; 11 men and five women) without cancer on standard and dual-energy subtraction radiographs were selected by approximate matching of age and sex to the 19 patients with cancer, and their examinations were performed on the same radiography system as the subjects with cancer. Some of the 16 dual-energy subtraction radiographs without missed lung cancers included incidental abnormal findings (other than cancer) similar to those seen in the selected cancer patients, including diffuse lung diseases, pleural abnormalities, and very small calcified benign nodules; the results were confirmed by CT.

All radiographs with and without missed cancers used in our observer study were obtained with a single-exposure dual-energy subtraction computed radiography system (FCR 5501ES, Fuji Medical Systems) using 110 kVp and 2.5–16 mAs, according to patient size.

Observer Study
Two 2-megapixel monochrome LCD monitors (MDL 2004A, Totoku Electric), similar to those used in our clinical reading area, were used in the observer study. Observers were advised that cases in the study might show no cancers, unilateral cancers, or bilateral cancers. However, observers were not informed of the number of patients with or the number of patients without cancers or that some of the cancers had been missed on the initial reading. Each case was scored first without review of the dual-energy subtraction images and then with the dual-energy subtraction images. Confidence bars were shown, one under each lung, both without and with dual-energy subtraction. Radiologists indicated their confidence level regarding the presence of a lung cancer, and if they thought a cancer was present, also marked the most likely position for each lung, first by use of standard PA and lateral chest radiographs and then with dual-energy subtraction images (soft-tissue and bone images). Windowing values were controlled manually by the radiologists, and a magnification tool was available. Two clinical parameters (patient age and sex) were provided on one of the monitors.

The six radiologists who participated in the observer study consisted of one chest radiologist (26 years of experience), three general radiologists (mean experience, 14 years; range, 10–16 years), and two radiology residents (third and fourth year). Radiologists were instructed regarding the nature of the task and appropriate use of the confidence rating scales. We provided a training session before the test with five cases that were not used in the study so that radiologists could learn how to operate the interface. The five cases included two representative examples with a nodular cancer and three without cancer.

Data Analysis
The confidence level ratings from each observer for 35 patients (70 lungs) were analyzed using receiver operating characteristic (ROC) methodology, and a quasi-maximum-likelihood estimation of the binormal distribution was fitted to the radiologists' confidence ratings [9]. The statistical significance of the difference in Az values (area under the ROC curve) between observers' readings without and with dual-energy subtraction images was tested using the Dorfman-Berbaum-Metz method [10], which included both reader variation and case sample variation by means of an analysis-of-variance approach. The quantity Az represents the area under an ROC curve that has been fit by use of the conventional binormal model. Because ROC curve area can be interpreted as sensitivity averaged over all specificities, larger values of Az indicate greater diagnostic accuracy [11, 12].

A marked location on an image was determined to be a correct localization (used to calculate sensitivity) if the mark point was located within the cancer boundary. Any marked locations that did not satisfy this criterion were considered to be incorrect localizations (false-positives). The sensitivity and number of false-positives were defined on the basis of the number of localizations by the observers regardless of the confidence level ratings. The statistical significance of the difference in correct or incorrect localizations between radiologists without and with the dual-energy subtraction images was estimated by use of the paired Student's t test for the six radiologists.


Results
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
Evaluation by ROC Analysis
The average Az value for the six radiologists was improved from 0.718 to 0.816, a statistically significant degree (p = 0.004) with the aid of dual-energy subtraction images (Table 1 and Fig. 2).


View this table:
[in this window]
[in a new window]

 
TABLE 1: Areas Under the Receiver Operating Characteristic Curve (Az) for the Detection of 19 Missed Cancers with Six Observers

 

Figure 2
View larger version (12K):
[in this window]
[in a new window]
[as a PowerPoint slide]
 
Fig. 2 Receiver operating characteristic (ROC) curves for detection of cancers on chest radiographs with six radiologists. Average area under ROC curve (Az) value was improved significantly from 0.718 to 0.816 with aid of dual-energy subtraction (ES) images.

 

Evaluation by Analysis of Localization
The average sensitivity (based on correct localizations) for 19 missed cancers with six radiologists was also significantly improved from 40% to 59% (p = 0.008) with dual-energy subtraction images (Table 2 and Fig. 3). The average number of false-positive (incorrect) localizations on 70 lungs was 10 without and nine with dual-energy subtraction images (p = 0.785) (Table 3).


View this table:
[in this window]
[in a new window]

 
TABLE 2: Number of Correct Localizations for 19 Missed Cancers with Six Observers

 

Figure 3
View larger version (16K):
[in this window]
[in a new window]
[as a PowerPoint slide]
 
Fig. 3 Bar graph shows number of correct localizations for 19 cancers by six radiologists with dual-energy subtraction images (gray bars) and without dual-energy subtraction images (white bars). For nine of 19 cancers (index numbers 4–6, 9–11, 14, 15, and 17), dual-energy subtraction image had beneficial effect for one or more radiologists. Dual-energy subtraction image had detrimental effect for only one cancer (number 12).

 

View this table:
[in this window]
[in a new window]

 
TABLE 3: Number of Incorrect Localizations (False-Positives) on 70 Lungs with Six Observers

 

The six radiologists marked a total of 46 cancer locations on the standard PA images and 67 cancer locations on the soft-tissue images. The radiologists also marked a total of 60 false-positive locations on the standard PA images and 53 false-positive locations on the soft-tissue images. Among these false-positives, three pulmonary vessels (one in the periphery and two in the hilum) were marked by two radiologists on both standard and soft-tissue images; one nipple was marked by five radiologists on the standard image and by four radiologists on the soft-tissue image; one bone lesion was marked by four radiologists only on the standard image; one peripheral vessel, one nipple, and one artifact were marked by two or three radiologists only on the soft-tissue image; and each of the remaining 81 false-positives was marked by only one radiologist on either the standard (n = 45) or the soft-tissue (n = 36) images.

Examples of Images in Observer Study
Many of the missed lung cancers were very subtle on the standard radiographs but were relatively obvious on the soft-tissue dual-energy subtraction images, and most radiologists detected these cancers with the dual-energy subtraction images (Figs. 4A, 4B, 5A, 5B, and 5C). Review of the dual-energy subtraction image had a detrimental effect for the detection of only one cancer (Figs. 6A and 6B).


Figure 4
View larger version (173K):
[in this window]
[in a new window]
[as a PowerPoint slide]
 
Fig. 4A Dual-energy subtraction radiograph in 60-year-old woman (number 5 in Fig. 3) with previously missed bronchoalveolar carcinoma in left lung. Missed cancer (arrows) is very subtle on standard posteroanterior image (A) but is relatively obvious on soft-tissue image (B). Note that no radiologists detected missed cancer without dual-energy subtraction and five radiologists detected it with dual-energy subtraction in this observer study.

 

Figure 5
View larger version (161K):
[in this window]
[in a new window]
[as a PowerPoint slide]
 
Fig. 4B Dual-energy subtraction radiograph in 60-year-old woman (number 5 in Fig. 3) with previously missed bronchoalveolar carcinoma in left lung. Missed cancer (arrows) is very subtle on standard posteroanterior image (A) but is relatively obvious on soft-tissue image (B). Note that no radiologists detected missed cancer without dual-energy subtraction and five radiologists detected it with dual-energy subtraction in this observer study.

 

Figure 6
View larger version (179K):
[in this window]
[in a new window]
[as a PowerPoint slide]
 
Fig. 5A Dual-energy subtraction radiographs in 66-year-old woman (number 10 in Fig. 3) with missed squamous cell carcinoma in left upper lobe. Standard posteroanterior image shows cancer (black arrow) and another nodular lesion (white arrow).

 

Figure 7
View larger version (166K):
[in this window]
[in a new window]
[as a PowerPoint slide]
 
Fig. 5B Dual-energy subtraction radiographs in 66-year-old woman (number 10 in Fig. 3) with missed squamous cell carcinoma in left upper lobe. Soft-tissue image (B) clearly shows missed cancer (arrow, B) and bone image (C) shows that other lesion is a fracture callus (arrow, C). Note that two radiologists detected cancer without dual-energy subtraction image, whereas five radiologists (three more) detected it with dual-energy subtraction image in this observer study.

 

Figure 8
View larger version (186K):
[in this window]
[in a new window]
[as a PowerPoint slide]
 
Fig. 5C Dual-energy subtraction radiographs in 66-year-old woman (number 10 in Fig. 3) with missed squamous cell carcinoma in left upper lobe. Soft-tissue image (B) clearly shows missed cancer (arrow, B) and bone image (C) shows that other lesion is a fracture callus (arrow, C). Note that two radiologists detected cancer without dual-energy subtraction image, whereas five radiologists (three more) detected it with dual-energy subtraction image in this observer study.

 

Figure 9
View larger version (175K):
[in this window]
[in a new window]
[as a PowerPoint slide]
 
Fig. 6A Dual-energy subtraction radiographs in 61-year-old woman (number 12 in Fig. 3) with primary adenocarcinoma in left lower lobe. Cropped views of left lung show missed cancers (arrows) on standard posteroanterior image (A) and soft-tissue image (B). Note that three radiologists detected cancer without dual-energy subtraction image, but none confirmed it with dual-energy subtraction (detrimental effect). This was probably because nodule appears to represent confluence of vessels on dual-energy subtraction soft-tissue image.

 

Figure 10
View larger version (155K):
[in this window]
[in a new window]
[as a PowerPoint slide]
 
Fig. 6B Dual-energy subtraction radiographs in 61-year-old woman (number 12 in Fig. 3) with primary adenocarcinoma in left lower lobe. Cropped views of left lung show missed cancers (arrows) on standard posteroanterior image (A) and soft-tissue image (B). Note that three radiologists detected cancer without dual-energy subtraction image, but none confirmed it with dual-energy subtraction (detrimental effect). This was probably because nodule appears to represent confluence of vessels on dual-energy subtraction soft-tissue image.

 


Discussion
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
Recent advances in chest radiography, including image processing, temporal subtraction, dual-energy subtraction imaging, and CAD, have been shown to enhance lung nodule detection [17, 1322]. However, it is important that available techniques are used appropriately in clinical practice to obtain maximum benefit. The lung cancers used in our observer study were previously missed by radiologists during routine interpretation even when dual-energy subtraction images were available. The observer study indicates that radiologists' performance for the detection of these cancers could have been improved by the use of dual-energy subtraction images, as indicated by the higher sensitivity, a similar number of false-positives, and improved ROC curves as compared with the use of the standard images alone. However, radiologists apparently did not review the dual-energy subtraction images in at least some instances, probably because they were not aware that dual-energy subtraction images were available. Dual-energy subtraction images are now displayed easily on our PACS, and review of these images requires only clicking on an icon or scrolling the mouse wheel to reveal them. At the time that many of these images were obtained, viewing dual-energy subtraction images required opening a separate application, and these images were viewed only selectively when the radiologist suspected an abnormality.

Studies have found that small nodular lung cancers are frequently missed on conventional chest radiographs by radiologists in clinical practice [2330]. Shah et al. [28] reported a missed cancer series with 40 cancers, in which 26 (65%) were obscured by two or three bones, nine (22%) were obscured by a clavicle or one or more ribs, and 12 (30%) were obscured by one bone. A similar incidence of obscuration by bone was apparent in another missed cancer series that we performed [8]. For those patients with dual-energy subtraction images, many of the cancers were very subtle on the standard radiographs but were relatively obvious on the soft-tissue dual-energy subtraction images. The radiologists' performance for the detection of missed cancers in our series was significantly improved (Az values improved from 0.718 to 0.816; detection rates, from 40% to 59%), and we believe soft-tissue images were responsible for the improvement because they minimize obscuration by bones.

Ide et al. [31] performed a study using 77 consecutive lung cancers and 77 healthy subjects without regard to "missed" or "visible" criteria and reported that Az values with five observers were increased from 0.46 to 0.53 for nonsolid, 0.72 to 0.82 for partly solid, and 0.97 to 0.99 for solid cancers with dual-energy subtraction images. The three cancer patterns were confirmed on thin-section CT performed at approximately the same time as the dual-energy subtraction studies. Their results suggest that some of these cancers were probably not visible on the radiograph, because the Az values were approximately 0.5 without and with dual-energy subtraction images for nonsolid cancers, and no localizations (to determine detection rate) were requested of observers in that study [31]. Our observer performance study used missed cancers that were identified in retrospect as visible on chest radiographs, although the CT studies used for confirmation were not always obtained at the same time as the dual-energy subtraction radiographs. In addition, we obtained the observers' localizations for cancers and false-positives, whereas the previous study did not.

A limitation of our study was that it was a retrospective observer study of a small number of patients with nodular cancers. Patients were identified from the University of Chicago Cancer Registry, and inclusion was based on the existence of an energy subtraction chest radiograph in which a peripheral cancer was visible in retrospect that had not been mentioned in the radiology report. Central hilar cancers were not included.

As with most observer tests, the results were influenced by the high incidence of cancers as compared with routine clinical practice and by the small number of cases. It is likely that the sensitivity was higher (increased detection rate of subtle cancers) and the specificity lower (increased false-positives) on both standard and dual-energy subtraction images in the observer study because observers were more focused on the specific task of detecting lung cancers than they might be in routine clinical practice. Although the detection rate of missed cancers in our series was significantly improved from 40% to 59% by use of dual-energy subtraction images in the observer study, approximately 40% of these small cancers were still missed because many were extremely subtle and difficult to detect.

In the observer performance study, we found the use of dual-energy subtraction images yielded an improvement in sensitivity and did not produce a detrimental effect for observers in terms of false-positive detections. We conclude that dual-energy subtraction chest radiography can substantially improve radiologists' ability to detect small lung cancers, and we believe that dual-energy subtraction images should be viewed routinely, rather than selectively, to derive maximum benefit.


Acknowledgments
 
We are grateful to Hiroyuki Abe, Jarvis Chen, Philip Caligiuri, Rodney Corby, Christopher Straus, and Yonglin Pu for participating as observers.


References
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 

  1. Freedman M. State-of-art screening for lung cancer. Part 1. The chest radiograph. Thorac Surg Clin 2004;14 : 43-52[CrossRef][Medline]
  2. McAdams HP, Samei E, Dobbins J III, Tourassi GD, Ravin CE. Recent advances in chest radiography. Radiology2006; 241:663 -683[Abstract/Free Full Text]
  3. Fraser RG, Hickey NM, Niklason LT, et al. Calcification in pulmonary nodules: detection with dual-energy digital radiography. Radiology 1986;160 : 595-601[Abstract/Free Full Text]
  4. Ergun DL, Mistretta CA, Brown DE, et al. Single-exposure dual-energy computed radiography: improved detection and processing. Radiology 1990;174 : 243-249[Abstract/Free Full Text]
  5. Ishigaki T, Sakuma S, Ikeda M. One-shot dual-energy subtraction chest imaging with computed radiography: clinical evaluation of film images. Radiology 1988;168 : 67-72[Abstract/Free Full Text]
  6. Kelcz F, Zink FE, Peppler WW, Kruger DG, Ergun DL, Mistretta CA. Conventional chest radiography vs dual-energy computed radiography in the detection and characterization of pulmonary nodules. AJR 1994; 164:271 -278
  7. Kido S, Ikezoe J, Naito H, et al. Clinical evaluation of pulmonary nodules with single-exposure dual-energy subtraction chest radiography with an iterative noise-reduction algorithm. Radiology1995; 194:407 -412[Abstract/Free Full Text]
  8. Li F, Engelmann R, Metz CE, Doi K, MacMahon H. Lung cancers missed on chest radiographs: results obtained with a commercial computer-aided detection program. Radiology 2008;264 : 273-280
  9. Metz CE, Herman BA, Shen JH. Maximum-likelihood estimation of receiver operating (ROC) curves from continuously distributed data. Stat Med 1998; 17:1033 -1053[CrossRef][Medline]
  10. Dorfman DD, Berbaum KS, Metz CE. ROC rating analysis: generalization to the population of readers and cases with the jackknife method. Invest Radiol 1992;27 : 723-731[CrossRef][Medline]
  11. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143 : 29-36[Abstract/Free Full Text]
  12. Metz CE. ROC methodology in radiologic imaging. Invest Radiol 1986; 21:720 -733[Medline]
  13. Correa J, Souto M, Tahoces PG, et al. Digital chest radiography: comparison of unprocessed and processed images in the detection of solitary pulmonary nodules. Radiology 1995;195 : 253-258[Abstract/Free Full Text]
  14. Woodard PK, Slone RM, Sagei SS, et al. Detection of CT-proved pulmonary nodules: comparison of selenium-based digital and conventional screen-film chest radiographs. Radiology1998; 209:705 -709[Abstract/Free Full Text]
  15. Difazio M, MacMahon H, Xu XW, et al. Digital chest radiography: effect of temporal subtraction images on detection accuracy. Radiology 1997;202 : 447-452[Abstract/Free Full Text]
  16. Kakeda S, Nakamura K, Kamada K, et al. Improved detection of lung nodules by using a temporal subtraction technique. Radiology 2002;224 : 145-151[Abstract/Free Full Text]
  17. Kakeda S, Kamada K, Hatakeyama Y, et al. Effect of temporal subtraction technique on interpretation time and diagnostic accuracy of chest radiography. AJR 2006;187 : 1253-1259[Abstract/Free Full Text]
  18. Tsukuda S, Heshiki A, Katsuragawa S, Li Q, MacMahon H, Doi K. Detection of lung nodules on digital chest radiographs: potential usefulness of a new contralateral subtraction technique. Radiology 2002;223 : 199-203[Abstract/Free Full Text]
  19. Doi K. Current status and future potential of computer-aided diagnosis in medical imaging. Br J Radiol2005; 78:3 -19[Abstract/Free Full Text]
  20. MacMahon H, Engelmann R, Behlen FM, et al. Computer-aided diagnosis of pulmonary nodules: results of a large-scale observer test. Radiology 1999;213 : 723-726[Abstract/Free Full Text]
  21. Johkoh T, Kozuka T, Tomiyama N, et al. Temporal subtraction for detection of solitary pulmonary nodules on chest radiographs: evaluation of a commercially available computer-aided diagnosis system. Radiology 2002;223 : 806-811[Abstract/Free Full Text]
  22. Kakeda S, Moriya J, Sato H, et al. Improved detection of lung nodules on chest radiographs using a commercial computer-aided diagnosis system. AJR 2004;182 : 505-510[Abstract/Free Full Text]
  23. Kundel HL. Predictive value and threshold detectability of lung tumors. Radiology 1981;139 : 25-29[Abstract/Free Full Text]
  24. Muhm JR, Miller WE, Fontana RS, Sanderson DR, Uhlenhopp MA. Lung cancer detected during a screening program using four-month chest radiographs. Radiology 1983;148 : 609-615[Abstract/Free Full Text]
  25. Heelan RT, Flehinger BJ, Melamed MR, et al. Nonsmall-cell lung cancer: results of the New York screening program. Radiology 1984;151 : 289-293[Abstract/Free Full Text]
  26. Quekel LGBA, Kessels AGH, Goei R, van Engelshoven JMA. Miss rate of lung cancer on the chest radiograph in clinical practice. Chest 1999; 115:720 -724[CrossRef][Medline]
  27. Austin JHM, Romney BM, Goldsmith LS. Missed bronchogenic carcinoma: radiographic findings in 27 patients with potentially resectable lesion evident in retrospect. Radiology 1992;182 : 115-122[Abstract/Free Full Text]
  28. Shah PK, Austin JHM, White CS, et al. Missed non–small cell lung cancers: radiographic findings of potentially respectable lesions evident only in retrospect. Radiology 2003;226 : 235-241[Abstract/Free Full Text]
  29. Monnier-Cholley L, Arrivé L, Porcel A, et al. Characteristics of missed lung cancer on chest radiographs: a French experience. Eur Radiol 2001;11 : 597-605[CrossRef][Medline]
  30. Monnier-Cholley L, Carrat F, Cholley BP, Tubiana JM, Arrivé L. Detection of lung cancer on radiographs: receiver operating characteristic analysis of radiologists', pulmonologists' and anesthes iologists' performance. Radiology 2004;233 : 799-805[Abstract/Free Full Text]
  31. Ide K, Mogami H, Murakami T, Yasuhara Y, Miyagawa M, Mochizuki T. Detection of lung cancer using single-exposure dual-energy subtraction chest radiography. Radiat Med 2007;25 : 195-201[CrossRef][Medline]

Add to CiteULike CiteULike   Add to Complore Complore   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Reddit Reddit   Add to Technorati Technorati    What's this?


This article has been cited by other articles:


Home page
Am. J. Roentgenol.Home page
S. Oda, K. Awai, K. Suzuki, Y. Yanaga, Y. Funama, H. MacMahon, and Y. Yamashita
Performance of Radiologists in Detection of Small Pulmonary Nodules on Chest Radiographs: Effect of Rib Suppression With a Massive-Training Artificial Neural Network
Am. J. Roentgenol., November 1, 2009; 193(5): W397 - W402.
[Abstract] [Full Text] [PDF]


This Article
Right arrow Abstract Freely available
Right arrow Figures Only
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Li, F.
Right arrow Articles by MacMahon, H.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Li, F.
Right arrow Articles by MacMahon, H.
Social Bookmarking
 Add to CiteULike   Add to Complore   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Reddit   Add to Technorati  
What's this?
Hotlight (NEW!)
Right arrow
What's Hotlight?


HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS