| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Original Research |
1 All authors: Kurt Rossmann Laboratories for Radiologic Image Research and the Department of Radiology, The University of Chicago, 5841 S Maryland Ave., MC-2026, Chicago, IL 60637.
Received July 16, 2007;
accepted after revision October 9, 2007.
This work was supported in part by USPHS grants CA61625, CA09119, and
EB00341 and by a research grant from Riverain Medical.
Abstract
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
MATERIALS AND METHODS. Dual-energy subtraction chest radiographs of 19 patients with previously missed nodular cancers, in which the radiology report did not mention a nodule that was visible in retrospect, were selected. Dual-energy subtraction radiographs of 19 patients with cancer and 16 patients without cancer were used for an observer study. Six radiologists indicated their confidence level regarding the presence of a lung cancer and, if they thought a cancer was present, also marked the most likely position for each lung, first using standard posteroanterior and lateral chest radiographs and then using both soft-tissue and bone dual-energy subtraction images along with standard radiographs. Receiver operating characteristic (ROC) curves were used to evaluate the observers' performance. The indicated locations of cancers and false-positives were also analyzed.
RESULTS. The average area under the ROC curve (Az) value for the six radiologists was improved from 0.718 to 0.816, a statistically significant amount (p = 0.004), and the average sensitivity (correct localizations) for 19 previously missed cancers was also significantly improved from 40% to 59% (p = 0.008) with the aid of dual-energy subtraction images. The average number of false-positive (incorrect) localizations on 70 lungs was 10 without and nine with dual-energy subtraction images (p = 0.785).
CONCLUSION. Dual-energy subtraction chest radiography has the potential to improve radiologists' performance for the detection of small missed lung cancers.
Keywords: diagnostic radiology • digital radiography • dual-energy subtraction radiography • lung neoplasms • observer performance study
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Dual-energy subtraction chest radiography has been used routinely for more than 10 years in our institution. We recently studied lung cancers previously missed by radiologists on chest radiographs in 34 patients [8], over half of whom also had dual-energy subtraction images. We noted that many of these lung cancers were very subtle on the standard radiographs but were relatively obvious on the soft-tissue images. We suspect that the radiologists did not review the dual-energy subtraction images in many such cases either because they were unaware that dual-energy subtraction images were available or because the dual-energy subtraction images were not easily available for viewing due to technical issues at that time. The purpose of our observer study was to evaluate whether dual-energy subtraction chest radiographs can improve radiologists' performance for the detection of small lung cancers.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Patient Database and Reference Standard
The records of the cancer registry at the University of Chicago Hospitals
were reviewed to identify all patients diagnosed with lung cancer from January
2001 to November 2004 (n = 821). All patients with chest radiographs
on file before treatment were reviewed, and the relevant radiographs and
reports were analyzed (n = 314). From the 314 patients, 34 standard
posteroanterior (PA) digital radiographs of 34 patients were identified in
which a nodular cancer was present on the standard PA image in retrospect, but
had not been mentioned in the initial report. The location of each missed
cancer on the 34 standard PA chest radiographs was identified by consensus of
two radiologists, with all confirmed by CT. All of the cancers were confirmed
using biopsy or surgery as the reference standard. The two radiologists had 15
and 30 years of experience, respectively, in interpreting chest radiographs.
Analysis of the 34 missed lung cancers on 34 standard PA chest radiographs has
been reported previously in a study of computer-aided detection (CAD)
[8].
Dual-Energy Subtraction Chest Radiographs for Observer Study
Among the 34 patients with missed lung cancers, 21 had dual-energy
subtraction radiographs, of which 19 were available for review. All of these
missed cancers were visible in retrospect on the standard PA images by
consensus of the two radiologists, and all were visible on the dual-energy
subtraction images as well. Among these 19 patients, three had two primary
cancers in different lungs. One patient had two lung cancers, both of which
were identifiable on the soft-tissue image, but only one of which was
identifiable on the standard image. Two patients had a small nodular cancer
(< 20 mm) that had been mentioned in the report in addition to the missed
cancer. These two detected cancers and the one that was identifiable only on
the dual-energy subtraction image were excluded from the analysis of observer
results, although the images were included in the observer test.
In total, 19 patients with missed lung cancers on standard (Fig. 1) or dual-energy subtraction radiographs (or both) were used in our observer study. The 19 patients with missed cancers included 12 men and seven women with a mean age of 69 years (range, 58–87 years). The mean interval between CT and corresponding dual-energy subtraction radiography for the 19 patients was 11 months (five within 1 month, six at 2–12 months, five at 13–24 months, and three at 25–32 months). Eight lung cancers were located in the right lung (four in the right upper lobe, one in the right middle lobe, and three in the right lower lobe), and 11 were located in the left lung (seven in the left upper lobe and four in the left lower lobe). The pathologic diagnoses for the 19 cancers included 17 non–small cell carcinomas (including seven adenocarcinomas, seven squamous cell carcinomas, and three unspecified non–small cell carcinomas) and two small cell carcinomas.
|
Before the observer study, the 19 missed cancers were graded for subtlety (from extremely subtle to extremely obvious on a 1–10 scale) on the standard images by the same two radiologists independently, and their numeric ratings were averaged. The cases consisted of 12 with ratings of 1 to 3 (extremely to very subtle) and seven with ratings of 3.5 to 6.5 (subtle to relatively obvious). The mean diameter (average of the length and width) of the 19 identifiable lung cancers was 15.6 mm (range, 7–22 mm) on standard images as measured by one radiologist.
Sixteen control subjects (mean age, 66 years; range, 50–79 years; 11 men and five women) without cancer on standard and dual-energy subtraction radiographs were selected by approximate matching of age and sex to the 19 patients with cancer, and their examinations were performed on the same radiography system as the subjects with cancer. Some of the 16 dual-energy subtraction radiographs without missed lung cancers included incidental abnormal findings (other than cancer) similar to those seen in the selected cancer patients, including diffuse lung diseases, pleural abnormalities, and very small calcified benign nodules; the results were confirmed by CT.
All radiographs with and without missed cancers used in our observer study were obtained with a single-exposure dual-energy subtraction computed radiography system (FCR 5501ES, Fuji Medical Systems) using 110 kVp and 2.5–16 mAs, according to patient size.
Observer Study
Two 2-megapixel monochrome LCD monitors (MDL 2004A, Totoku Electric),
similar to those used in our clinical reading area, were used in the observer
study. Observers were advised that cases in the study might show no cancers,
unilateral cancers, or bilateral cancers. However, observers were not informed
of the number of patients with or the number of patients without cancers or
that some of the cancers had been missed on the initial reading. Each case was
scored first without review of the dual-energy subtraction images and then
with the dual-energy subtraction images. Confidence bars were shown, one under
each lung, both without and with dual-energy subtraction. Radiologists
indicated their confidence level regarding the presence of a lung cancer, and
if they thought a cancer was present, also marked the most likely position for
each lung, first by use of standard PA and lateral chest radiographs and then
with dual-energy subtraction images (soft-tissue and bone images). Windowing
values were controlled manually by the radiologists, and a magnification tool
was available. Two clinical parameters (patient age and sex) were provided on
one of the monitors.
The six radiologists who participated in the observer study consisted of one chest radiologist (26 years of experience), three general radiologists (mean experience, 14 years; range, 10–16 years), and two radiology residents (third and fourth year). Radiologists were instructed regarding the nature of the task and appropriate use of the confidence rating scales. We provided a training session before the test with five cases that were not used in the study so that radiologists could learn how to operate the interface. The five cases included two representative examples with a nodular cancer and three without cancer.
Data Analysis
The confidence level ratings from each observer for 35 patients (70 lungs)
were analyzed using receiver operating characteristic (ROC) methodology, and a
quasi-maximum-likelihood estimation of the binormal distribution was fitted to
the radiologists' confidence ratings
[9]. The statistical
significance of the difference in Az values (area under
the ROC curve) between observers' readings without and with dual-energy
subtraction images was tested using the Dorfman-Berbaum-Metz method
[10], which included both
reader variation and case sample variation by means of an analysis-of-variance
approach. The quantity Az represents the area under an ROC
curve that has been fit by use of the conventional binormal model. Because ROC
curve area can be interpreted as sensitivity averaged over all specificities,
larger values of Az indicate greater diagnostic accuracy
[11,
12].
A marked location on an image was determined to be a correct localization (used to calculate sensitivity) if the mark point was located within the cancer boundary. Any marked locations that did not satisfy this criterion were considered to be incorrect localizations (false-positives). The sensitivity and number of false-positives were defined on the basis of the number of localizations by the observers regardless of the confidence level ratings. The statistical significance of the difference in correct or incorrect localizations between radiologists without and with the dual-energy subtraction images was estimated by use of the paired Student's t test for the six radiologists.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
|
|
Evaluation by Analysis of Localization
The average sensitivity (based on correct localizations) for 19 missed
cancers with six radiologists was also significantly improved from 40% to 59%
(p = 0.008) with dual-energy subtraction images
(Table 2 and
Fig. 3). The average number of
false-positive (incorrect) localizations on 70 lungs was 10 without and nine
with dual-energy subtraction images (p = 0.785)
(Table 3).
|
|
|
The six radiologists marked a total of 46 cancer locations on the standard PA images and 67 cancer locations on the soft-tissue images. The radiologists also marked a total of 60 false-positive locations on the standard PA images and 53 false-positive locations on the soft-tissue images. Among these false-positives, three pulmonary vessels (one in the periphery and two in the hilum) were marked by two radiologists on both standard and soft-tissue images; one nipple was marked by five radiologists on the standard image and by four radiologists on the soft-tissue image; one bone lesion was marked by four radiologists only on the standard image; one peripheral vessel, one nipple, and one artifact were marked by two or three radiologists only on the soft-tissue image; and each of the remaining 81 false-positives was marked by only one radiologist on either the standard (n = 45) or the soft-tissue (n = 36) images.
Examples of Images in Observer Study
Many of the missed lung cancers were very subtle on the standard
radiographs but were relatively obvious on the soft-tissue dual-energy
subtraction images, and most radiologists detected these cancers with the
dual-energy subtraction images (Figs.
4A,
4B,
5A,
5B, and
5C). Review of the dual-energy
subtraction image had a detrimental effect for the detection of only one
cancer (Figs. 6A and
6B).
|
|
|
|
|
|
|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Studies have found that small nodular lung cancers are frequently missed on conventional chest radiographs by radiologists in clinical practice [23–30]. Shah et al. [28] reported a missed cancer series with 40 cancers, in which 26 (65%) were obscured by two or three bones, nine (22%) were obscured by a clavicle or one or more ribs, and 12 (30%) were obscured by one bone. A similar incidence of obscuration by bone was apparent in another missed cancer series that we performed [8]. For those patients with dual-energy subtraction images, many of the cancers were very subtle on the standard radiographs but were relatively obvious on the soft-tissue dual-energy subtraction images. The radiologists' performance for the detection of missed cancers in our series was significantly improved (Az values improved from 0.718 to 0.816; detection rates, from 40% to 59%), and we believe soft-tissue images were responsible for the improvement because they minimize obscuration by bones.
Ide et al. [31] performed a study using 77 consecutive lung cancers and 77 healthy subjects without regard to "missed" or "visible" criteria and reported that Az values with five observers were increased from 0.46 to 0.53 for nonsolid, 0.72 to 0.82 for partly solid, and 0.97 to 0.99 for solid cancers with dual-energy subtraction images. The three cancer patterns were confirmed on thin-section CT performed at approximately the same time as the dual-energy subtraction studies. Their results suggest that some of these cancers were probably not visible on the radiograph, because the Az values were approximately 0.5 without and with dual-energy subtraction images for nonsolid cancers, and no localizations (to determine detection rate) were requested of observers in that study [31]. Our observer performance study used missed cancers that were identified in retrospect as visible on chest radiographs, although the CT studies used for confirmation were not always obtained at the same time as the dual-energy subtraction radiographs. In addition, we obtained the observers' localizations for cancers and false-positives, whereas the previous study did not.
A limitation of our study was that it was a retrospective observer study of a small number of patients with nodular cancers. Patients were identified from the University of Chicago Cancer Registry, and inclusion was based on the existence of an energy subtraction chest radiograph in which a peripheral cancer was visible in retrospect that had not been mentioned in the radiology report. Central hilar cancers were not included.
As with most observer tests, the results were influenced by the high incidence of cancers as compared with routine clinical practice and by the small number of cases. It is likely that the sensitivity was higher (increased detection rate of subtle cancers) and the specificity lower (increased false-positives) on both standard and dual-energy subtraction images in the observer study because observers were more focused on the specific task of detecting lung cancers than they might be in routine clinical practice. Although the detection rate of missed cancers in our series was significantly improved from 40% to 59% by use of dual-energy subtraction images in the observer study, approximately 40% of these small cancers were still missed because many were extremely subtle and difficult to detect.
In the observer performance study, we found the use of dual-energy subtraction images yielded an improvement in sensitivity and did not produce a detrimental effect for observers in terms of false-positive detections. We conclude that dual-energy subtraction chest radiography can substantially improve radiologists' ability to detect small lung cancers, and we believe that dual-energy subtraction images should be viewed routinely, rather than selectively, to derive maximum benefit.
Acknowledgments
We are grateful to Hiroyuki Abe, Jarvis Chen, Philip Caligiuri, Rodney
Corby, Christopher Straus, and Yonglin Pu for participating as observers.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
This article has been cited by other articles:
|
|
 |
|
  S. Oda, K. Awai, K. Suzuki, Y. Yanaga, Y. Funama, H. MacMahon, and Y. Yamashita Performance of Radiologists in Detection of Small Pulmonary Nodules on Chest Radiographs: Effect of Rib Suppression With a Massive-Training Artificial Neural Network Am. J. Roentgenol., November 1, 2009; 193(5): W397 - W402. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
