Radiologists' Performance for Differentiating Benign from Malignant Lung Nodules on High-Resolution CT Using Computer-Estimated Likelihood of Malignancy
Feng Li1,
Masahito Aoyama2,
Junji Shiraishi1,
Hiroyuki Abe1,
Qiang Li1,
Kenji Suzuki1,
Roger Engelmann1,
Shusuke Sone3,
Heber MacMahon1 and
Kunio Doi1
1 Department of Radiology, Kurt Rossmann Laboratories for Radiologic Image
Research, MC-2026, The University of Chicago, 5841 S Maryland Ave., Chicago,
IL 60637.
2 Department of Intelligent Systems, Faculty of Information Sciences, Hiroshima
City University, Hiroshima 731-3194, Japan.
3 Azumi General Hospital, Ikeda, Nagano 399-8695, Japan.
View larger version (114K):
[in a new window]
|
Fig. 1A. —Radiologists' average ratings without and with computer
output for six cases used in observer study. Note that difference in
likelihood of malignancy between computer output and initial radiologists'
ratings was not large in cases shown here. Radiologists' interpretation with
computer-aided diagnosis (CAD) scheme was, in general, more accurate than
radiologists without CAD scheme in most malignant and benign nodules.
High-resolution CT (HRCT) scan of 55-year-old woman with lung cancer shows
pure ground-glass opacity. Computer output was 0.66; radiologists' ratings
without CAD, 0.64; and radiologists' ratings with CAD, 0.71.
|
|
View larger version (120K):
[in a new window]
|
Fig. 1B. —Radiologists' average ratings without and with computer
output for six cases used in observer study. Note that difference in
likelihood of malignancy between computer output and initial radiologists'
ratings was not large in cases shown here. Radiologists' interpretation with
computer-aided diagnosis (CAD) scheme was, in general, more accurate than
radiologists without CAD scheme in most malignant and benign nodules. HRCT
scan of 57-year-old woman with benign nodule shows pure ground-glass opacity.
Computer output was 0.24; radiologists' ratings without CAD, 0.32; and
radiologists' ratings with CAD, 0.27.
|
|
View larger version (126K):
[in a new window]
|
Fig. 1C. —Radiologists' average ratings without and with computer
output for six cases used in observer study. Note that difference in
likelihood of malignancy between computer output and initial radiologists'
ratings was not large in cases shown here. Radiologists' interpretation with
computer-aided diagnosis (CAD) scheme was, in general, more accurate than
radiologists without CAD scheme in most malignant and benign nodules. HRCT
scan of 73-year-old man with lung cancer shows mixed ground-glass opacity.
Computer output was 0.90; radiologists' ratings without CAD, 0.75; and
radiologists' ratings with CAD, 0.85.
|
|
View larger version (139K):
[in a new window]
|
Fig. 1D. —Radiologists' average ratings without and with computer
output for six cases used in observer study. Note that difference in
likelihood of malignancy between computer output and initial radiologists'
ratings was not large in cases shown here. Radiologists' interpretation with
computer-aided diagnosis (CAD) scheme was, in general, more accurate than
radiologists without CAD scheme in most malignant and benign nodules. HRCT
scan of 79-year-old man with benign nodule shows mixed ground-glass opacity.
Computer output was 0.57; radiologists' ratings without CAD, 0.48; and
radiologists' ratings with CAD, 0.56.
|
|
View larger version (155K):
[in a new window]
|
Fig. 1E. —Radiologists' average ratings without and with computer
output for six cases used in observer study. Note that difference in
likelihood of malignancy between computer output and initial radiologists'
ratings was not large in cases shown here. Radiologists' interpretation with
computer-aided diagnosis (CAD) scheme was, in general, more accurate than
radiologists without CAD scheme in most malignant and benign nodules. HRCT
scan of 57-year-old man with lung cancer shows solid opacity. Computer output
was 0.78; radiologists' ratings without CAD, 0.66; and radiologists' ratings
with CAD, 0.76.
|
|
View larger version (126K):
[in a new window]
|
Fig. 1F. —Radiologists' average ratings without and with computer
output for six cases used in observer study. Note that difference in
likelihood of malignancy between computer output and initial radiologists'
ratings was not large in cases shown here. Radiologists' interpretation with
computer-aided diagnosis (CAD) scheme was, in general, more accurate than
radiologists without CAD scheme in most malignant and benign nodules. HRCT
scan of 68-year-old man with benign nodule shows solid opacity. Computer
output was 0.36; radiologists' ratings without CAD, 0.37; and radiologists'
ratings with CAD, 0.36.
|
|
View larger version (24K):
[in a new window]
|
Fig. 2. —Graph shows receiver operating characteristic (ROC) curves
for performance of computer alone and average performance of 16 radiologists
without and with computer-aided diagnosis (CAD) scheme. Note that difference
was statistically significant between radiologists without and with CAD scheme
(p = 0.016), between computer alone and radiologists' performance
without (p = 0.00006), and between computer alone and radiologists'
performance with CAD scheme (p = 0.0005).
|
|
View larger version (126K):
[in a new window]
|
Fig. 4A. —High-resolution CT (HRCT) scans show one malignant case and
one benign case. Note that radiologists' interpretations without
computer-aided diagnosis (CAD) scheme were correct in these cases, whereas
likelihoods of malignancy based on computer outputs only were obviously
incorrect; even with incorrect CAD outputs, radiologists retained correct
ratings. HRCT scan shows malignant lung nodule in 68-year-old man. Computer
output was 0.36; radiologists' ratings without CAD, 0.67; and radiologists'
ratings with CAD, 0.61.
|
|
View larger version (152K):
[in a new window]
|
Fig. 4B. —High-resolution CT (HRCT) scans show one malignant case and
one benign case. Note that radiologists' interpretations without
computer-aided diagnosis (CAD) scheme were correct in these cases, whereas
likelihoods of malignancy based on computer outputs only were obviously
incorrect; even with incorrect CAD outputs, radiologists retained correct
ratings. HRCT scan shows benign lung nodule in 35-year-old woman. Computer
output was 0.78; radiologists' ratings without CAD, 0.27; and radiologists'
ratings with CAD, 0.38.
|
|
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
Copyright © 2004 by the American Roentgen Ray Society.
= average
radiologists' ratings for malignant nodules, 
= average radiologists'
ratings for benign nodules, horizontal lines = range of radiologists' ratings
for each nodule. Graphs show correlation between computer outputs and average
radiologists' ratings without CAD (A) (r = 0.514) and with CAD
(B) (r = 0.784). Note that radiologists' ratings without CAD
scheme in some malignant (upper left quadrant) and benign (lower
right quadrant) nodules were obviously correct, whereas likelihood of
malignancy based on computer outputs alone was incorrect; even with incorrect
CAD outputs, radiologists retained correct ratings. One malignant case
(arrow) and one benign case (arrowhead) shown here are
illustrated in Figures 4A,
4B.
