DOI:10.2214/AJR.05.1582
AJR 2006; 187:1483-1491
© American Roentgen Ray Society
Prospective Assessment of Computer-Aided Detection in Interpretation of Screening Mammography
Justin M. Ko1,
Michael J. Nicholas1,2,
Jeffrey B. Mendel1 and
Priscilla J. Slanetz1,3
1 Department of Radiology, Caritas St. Elizabeth's Medical Center and Tufts
University School of Medicine, Boston, MA.
2 Present address: Department of Radiology, Hospital of Saint Raphael, New
Haven, CT.
3 Present address: Department of Radiology, Boston University School of
Medicine, Boston Medical Center, 88 East Newton St., Boston, MA 02118.
Received September 7, 2005;
accepted after revision February 14, 2006.
Address correspondence to P. J. Slanetz
(priscilla.slanetz{at}bmc.org).
Abstract
OBJECTIVE. The purpose of this study was to prospectively assess the
usefulness of computer-aided detection (CAD) in the interpretation of
screening mammography and to provide the true sensitivity and specificity of
this technique in a clinical setting.
SUBJECTS AND METHODS. Over a 26-month period, 5,016 screening
mammograms were interpreted without, and subsequently with, the assistance of
the iCAD MammoReader detection system. Data collected for actionable findings
included dominant feature (calcification, mass, asymmetry, architectural
distortion), detection method (radiologist only, CAD only, or both radiologist
and CAD), BI-RADS assessment code, associated histopathology for those
undergoing biopsy, and tumor stage for malignant lesions. The study population
was cross-checked against an independent reference standard to identify
false-negative cases.
RESULTS. Of the 5,016 cases, the recall rate increased from 12% to
14% with the addition of CAD. Of the 107 (2%) patients who underwent biopsy,
101 (94%) were prompted by the radiologist and six (6%) were prompted by CAD.
Of the 124 biopsies performed on actionable findings in the 107 patients,
findings in 79 (64%) were benign and in 45 (36%) were in situ or invasive
carcinoma. Three study participants who were not recalled by the radiologist
with the assistance of CAD developed cancer within 1 year of the screening
mammogram and were considered to be false-negative cases. The radiologist
detected 43 (90%) of the 48 total malignancies and 45 (94%) of the 48
malignancies with the assistance of CAD. CAD missed eight cancers that were
detected by the radiologist, which presented as architectural distortions
(n = 3), irregular masses (n = 4), and a circumscribed mass
(n = 1). CAD detected two in situ cancers as a faint cluster of
calcifications that had not been perceived by the radiologist and one mass
that was dismissed by the radiologist, accounting for at least a 4.7% increase
in cancer detection rate. Sensitivity of screening mammography with the use of
CAD (94%) represented an absolute and relative 4% increase over the
sensitivity of the radiologist alone (90%). Specificity of screening
mammography with and without the use of CAD was 99%.
CONCLUSION. Routine use of CAD while interpreting screening
mammograms significantly increases recall rates, has no significant effect on
positive predictive value for biopsy, and can increase cancer detection rate
by at least 4.7% and sensitivity by at least 4%. This study provides
"true" values for sensitivity and specificity for use of CAD in
interpretation of screening mammography as measured prospectively in the
context of a working clinical setting.
Keywords: breast cancer • computer-aided detection • mammography • screening
Introduction
There is mounting evidence that screening mammography has a significant
impact on reducing mortality through early detection of malignant lesions in
women older than 40 years [1,
2]. The effectiveness of
screening mammography in reducing mortality has thus far been limited by its
sensitivity, which several retrospective studies have shown to range from 80%
to 90%
[3-5].
The development and implementation of computer-aided detection (CAD) hold the
potential to improve screening mammography by marking suspicious findings
otherwise missed by radiologists, thus increasing detection rates and
sensitivity. This promise of CAD, together with U.S. Food and Drug
Administration (FDA) approvals and insurance reimbursement, have led an
increasing number of radiology practices to use this technology
[6] clinically.
Although there is a preponderance of evidence related to the usefulness of
CAD as measured retrospectively in a laboratory setting
[5,
7,
8], there are relatively few
studies that prospectively examine the use of CAD in a working clinical
environment
[9-11].
Furthermore, there are no studies that provide "true" sensitivity
and specificity of the use of CAD in the interpretation of screening
mammography in a working clinical setting. It is time-consuming and expensive
to measure prospectively the effect of the use of CAD, and the low incidence
of breast cancer in a screening population largely limits the statistical
significance of findings. Furthermore, measurement of sensitivity and
specificity requires long-term follow-up and comparison of study participants
against an independent reference source. Although retrospective studies can be
designed to have the statistical power to conclude about the significance of
findings, they are not true indicators of how CAD performs in the clinical
arena and do not provide measures of sensitivity and specificity of screening
mammography with the use of CAD in a working clinical setting.
The purpose of this article is to prospectively assess the clinical
usefulness of CAD in the interpretation of screening mammograms and to
determine its impact on recall rate, positive predictive value, cancer
detection rates, and the sensitivity and specificity of screening
mammography.
Subjects and Methods
This study included screening mammograms interpreted by two experienced
radiologists at Caritas St. Elizabeth's Medical Center between June 1, 2002
and July 31, 2004. Each of the two radiologists, one with 20 years of clinical
experience and the other with fellowship training in breast imaging and 9
years of clinical experience, accumulated 2 months of clinical experience with
the CAD system used in this study (iCAD MammoReader, iCAD, upgraded as
available from manufacturer: software versions 3.2 from June 2002 to February
2003, 3.6 from February 2003 to March 2003, and 3.9 from March 2003 to July
2004) before enrolling patients. This study was performed under an
institutional review board-approved protocol.
During the 26-month study period, a total of 5,016 asymptomatic women
undergoing screening mammography and CAD evaluation were enrolled in the
protocol. Standard screening guidelines were used, and high-risk patients
between the ages of 35 and 40 years were encouraged to undergo screening.
Patients with a history of breast carcinoma were accepted into the study as
long as they had been disease-free for 5 years.
A total of 4,835 (98%) bilateral and 123 (2%) unilateral mammograms were
obtained, with a standard two-view evaluation of each breast performed by a
mammography-certified radiology technologist. All were imaged using
Instrumentarium Performa or Diamond conventional film-screen systems (GE
Healthcare). The mammograms were subsequently digitized and analyzed by the
iCAD MammoReader system. This particular system uses two types of
marks—circles indicating areas of possible microcalcifications and
crosses indicating areas of possible masses or architectural distortion.
Each mammogram was first interpreted by the radiologist without knowledge
of CAD results. In cases in which prior films were available, the images were
compared with a study at least 2 years before the current study. If there was
a positive finding, the radiologist made a decision regarding recall or
follow-up before reviewing the CAD results. The digitized CAD-analyzed image
with associated marks was then reviewed, with the radiologist focusing on
areas marked by CAD. The radiologist noted any actionable findings, defined as
an abnormality on the screening study prompting the interpreting radiologist
to recall a patient for additional imaging, as a result of this review, and
made the additional recall decisions as necessary. In instances in which a
finding prompted radiologist recall before review of CAD analysis, the
radiologist marked the area on the printed CAD-analyzed image as follows:
"+" marks on a CAD-analyzed image indicated potential masses
identified by CAD; "O" marks indicated potential calcifications
identified by CAD; and crayon marks indicated areas prompting recall by the
interpreting radiologist. In this study, a radiologist could make the decision
to recall a patient on revaluation after viewing CAD results, but could not
change a recall decision to a negative assessment based on lack of CAD marking
in an area of potential abnormality.
Data collected for actionable findings included dominant feature
(calcification, mass, asymmetry, architectural distortion), detection method
(radiologist only, CAD only, or both radiologist and CAD), BI-RADS assessment
code, associated histopathology for those undergoing biopsy, and tumor stage
for malignant lesions. We did not systematically analyze lesion size or breast
density in our study. The study population was subsequently cross-checked
against the Caritas St. Elizabeth's Medical Center cancer registry of all
breast malignancies diagnosed between June 1, 2002 and July 31, 2005, to
determine the pool of patients who developed malignancy within 12 months of
screening mammography.
All data were entered into a Microsoft Excel spreadsheet for analysis, and
all statistical analysis was performed using the Statistical Analysis Toolpak
(Microsoft). A p value of less than 0.05 was considered to be
statistically significant.
Results
Of the 5,016 women screened during the study period, 696 were recalled
based on 816 actionable findings. Of these recalls, 607 were based on
radiologist interpretation before CAD analysis (727 actionable findings).
After reviewing the CAD analysis, the radiologist recalled an additional 89
patients (89 actionable findings).
As Table 1 shows, the use of
CAD increased the total number of recalls by 15% (from 607 to 696) and
increased the number of actionable findings by 12% (from 727 to 816). The
recall rate increased in absolute terms by 2% (from 12% to 14%), which
represented a statistically significant increase. The use of CAD increased the
number of actionable microcalcification findings by 17% (from 153 to 179) and
the number of actionable asymmetry and masses by 11% (from 574 to 637),
producing a slight shift in the proportion of microcalcifications and
asymmetry or masses deemed actionable.
Of the 696 patients recalled, 689 returned for subsequent diagnostic
evaluation and seven were lost to follow-up.
Table 2 reveals the effect of
CAD on the BI-RADS assessment of recalled patients. The absolute number of
patients categorized as BI-RADS category 1 or 2 increased 18% (from 391 to
462), patients categorized as BI-RADS category 3 increased 10% (from 103 to
113), and patients categorized as BI-RADS category 4 or 5 increased 6% (from
108 to 114). CAD effectively increased the proportion of patients with final
BI-RADS assessments of categories 1 and 2 from 64% to 66%, while decreasing
the proportion in BI-RADS categories 4 and 5 from 35% to 33%. Standard
recommendations were made based on the BI-RADS assessment.
Of the 114 women recommended for biopsy, seven did not return and were
deemed, for purposes of the study, lost to follow-up.
Table 3 outlines the
histopathologic characteristics for the 124 actionable findings in the
remaining 107 women who were recommended for biopsy and who underwent the
procedure. The effect of CAD was to decrease the proportion of malignant
lesions slightly (positive predictive value for biopsy) from 37% (43/117) to
36% (45/124), a statistically insignificant finding. The proportion of both
benign and malignant lesions by category shifted modestly as a result of the
effect of CAD. Furthermore, the use of CAD decreased the proportion of
patients with malignancies to patients recalled by 9% (from 6.8% to 6.2%) and
the proportion of malignant lesions to total actionable findings by 7% (from
5.9% to 5.5%).
As Table 4 shows, the two
additional lesions detected with the aid of CAD were stage 0 and stage I
tumors, increasing the proportion of early-stage (stage 0 or I) tumors
detected by 1% (from 72% to 73%). The malignant mass marked by CAD on one
view, but which was dismissed by the radiologist, was a stage II tumor.
Table 5 examines the effect
of CAD on the detection of malignant lesions. CAD increased detection rate of
malignant lesions by 4.7% (from 0.86% to 0.90%). The radiologist alone
detected 43 (90%) of 48 malignant lesions. The use of CAD added two
malignancies to increase the total to 45 (94%) of 48 malignant lesions. The
radiologist alone detected 90% (27/30) of malignant asymmetry or masses and
89% of malignant microcalcifications (16/18). The radiologist with the
assistance of CAD detected 90% (27/30) of malignant asymmetry or masses and
100% (18/18) of microcalcifications. CAD alone marked 67% (20/30) of asymmetry
or masses and 100% of microcalcifications (18/18). Of the 48 malignancies
detected, 77% (37/48) were initially noted by the radiologist and marked by
CAD. Two percent (1/48) were marked by CAD and dismissed by the radiologist,
and 4% (2/48) were not detected by either the radiologist or CAD.
The sensitivity and specificity of screening mammography both with and
without the use of CAD were also calculated. The radiologist alone had a
sensitivity of 90% (43/48 total malignant lesions detected), which with the
assistance of CAD increased by 4% in absolute and relative terms to 94%
(45/48). While not clinically relevant, CAD alone had a sensitivity of 79%
(38/48). The specificity for both the radiologist alone and with the
assistance of CAD was 99%. CAD missed eight cancers detected by the
radiologist, which presented as architectural distortions (n = 3),
irregular masses (n = 4), and a circumscribed mass (n = 1).
An additional two masses that were missed by both the radiologist and CAD
could not be observed on imaging and were categorized as false-negatives. CAD
detected two in situ cancers as a faint cluster of calcifications that had not
been perceived by the radiologist and one mass that was dismissed by the
radiologist, which was categorized as a false-negative.
View larger version (91K):
[in this window]
[in a new window]
[as a PowerPoint slide]
|
Fig. 1A —49-year-old woman underwent bilateral screening mammography.
Patient presented with palpable mass 5 months after screening mammography.
Stage II invasive ductal carcinoma was seen at biopsy. Mediolateral oblique
view reveals no suspicious findings.
|
|
View larger version (78K):
[in this window]
[in a new window]
[as a PowerPoint slide]
|
Fig. 1B —49-year-old woman underwent bilateral screening mammography.
Patient presented with palpable mass 5 months after screening mammography.
Stage II invasive ductal carcinoma was seen at biopsy. Craniocaudal view shows
subtle asymmetry.
|
|
View larger version (17K):
[in this window]
[in a new window]
[as a PowerPoint slide]
|
Fig. 1C —49-year-old woman underwent bilateral screening mammography.
Patient presented with palpable mass 5 months after screening mammography.
Stage II invasive ductal carcinoma was seen at biopsy. Right craniocaudal view
at time of screening mammogram.
|
|
View larger version (46K):
[in this window]
[in a new window]
[as a PowerPoint slide]
|
Fig. 1D —49-year-old woman underwent bilateral screening mammography.
Patient presented with palpable mass 5 months after screening mammography.
Stage II invasive ductal carcinoma was seen at biopsy. Right craniocaudal view
4 months later. Asymmetry was not marked by radiologist but was marked by
computer-aided detection (CAD) on screening mammogram craniocaudal view. Mass
(arrow) was not marked by CAD when patient returned with palpable
mass. Linear markers on images are routinely placed on skin over visible
incision sites in women who have previously undergone surgery.
|
|
The first 3,469 mammograms in the study (3,402 bilaterals and 67
unilaterals) were analyzed for marks placed by CAD to determine the
specificity of marks for CAD. The iCAD MammoReader System analyzed 13,742
images (four per bilateral mammogram and two per unilateral mammogram) and
placed 13,719 marks, for an average of 1.0 mark per image and 4.0 marks per
four-view examination. Of these 13,719 marks, 353 (3%) were deemed actionable
by the radiologist (true-positives) and the remaining 13,366 (97%) were
dismissed (false-positives).
Discussion
The objective of CAD technology is to improve the sensitivity of screening
mammography, which currently comes at the cost of low specificity. In the
first 3,469 mammograms analyzed, there was an average of 1.0 false-positive
mark per image and 4.0 false-positive marks per four-image examination. Of the
13,742 marks placed, only 353 were deemed actionable by the radiologist. The
large number of false-positive marks can potentially inhibit the efficacy of
CAD by over-whelming the radiologist interpreting the films. Other
investigators suggest that 1.3 false-positive marks per image is a value that
does not lead to unduly increased recall rates or time for interpretation
[12].
A true-positive mark by CAD does not directly translate to enhanced
radiologist performance with CAD in a clinical setting. Rather, it is the
interaction between the radiologist and the technology that produces increased
cancer detection and sensitivity. In our study, there was one case of CAD
marking a malignant lesion (mass) that was dismissed by the radiologist,
possibly influenced by the low specificity of the CAD system. The patient
presented 5 months later with a palpable mass (Figs.
1A,
1B,
1C, and
1D). Because no independent
standard was used in previous prospective trials
[9-11],
it is impossible to determine the number of such cases for comparison with CAD
specificity. As CAD systems evolve, the trade-off between sensitivity and
specificity may become less severe, and further research may elucidate the
association between CAD specificity and cancer detection rates.
View larger version (123K):
[in this window]
[in a new window]
[as a PowerPoint slide]
|
Fig. 2A —50-year-old woman underwent bilateral screening mammography.
Grade II ductal carcinoma in situ was seen at biopsy. Mediolateral oblique
(A) and craniocaudal (B) views show faint calcifications that
were marked by computer-aided detection (CAD) on craniocaudal view, as
indicated by irregular circle in left breast, outer quadrant, but were not
marked by radiologist.
|
|
View larger version (115K):
[in this window]
[in a new window]
[as a PowerPoint slide]
|
Fig. 2B —50-year-old woman underwent bilateral screening mammography.
Grade II ductal carcinoma in situ was seen at biopsy. Mediolateral oblique
(A) and craniocaudal (B) views show faint calcifications that
were marked by computer-aided detection (CAD) on craniocaudal view, as
indicated by irregular circle in left breast, outer quadrant, but were not
marked by radiologist.
|
|
At its core, an assessment of the use of CAD in screening mammography is
based on weighing the benefit of using CAD (i.e., an increased true-positive
rate) against its costs (i.e., an increased false-positive rate). Our study
design is largely similar to the one used by Freer and Ulissey
[9] and described in their
seminal article. Their study "duplicated the conditions of real-life
mammographic interpretation....The only way to improve on the study would be
to repeat it at numerous institutions, to be certain that others could match
their performance. The study design itself was flawless"
[13]. Our study is
differentiated from that of Freer and Ulissey in that it employs an
independent reference standard and performs long-term follow-up, which allows
the detection of false-negative cases and calculation of "true"
sensitivity and specificity.
In Freer and Ulissey's 2001 study
[9], a 19.5% increase in cancer
detection rate was associated with a statistically significant increase in
recall rate and no decrease in positive predictive value of biopsy. Gur et al.
[10] found no significant
increase in either cancer detection or recall rate. It is possible, however,
that an adjustment for patient characteristics would have resulted in modest
increases in recall and cancer detection rate
[14]. Birdwell et al.
[11] found a 7.4% increase in
cancer detection rate with a 1% increase in recall rate. The results from our
study indicate a 4.7% increase in cancer detection rate (from 0.86% to 0.90%),
a statistically significant increase in recall rate (12.1% to 13.9%), and no
significant decrease in positive predictive value of biopsy (36.7% to
36.2%).
In our study, an additional 89 women were recalled and an additional six
women biopsied to detect an additional two cancers, resulting in 44.5 recalls
and 3.0 biopsies for each additional cancer detected. We compared these
benchmarks to the results based on radiologist interpretation alone, in which
607 women were recalled and 101 women biopsied to detect 43 cancers, resulting
in 14.1 recalls and 2.3 biopsies for each cancer detected. To determine
whether the additional direct costs of the procedures and indirect costs
associated with anxiety, pain, and discomfort of recalled and biopsied
patients [15] as a result of
the use of CAD are offset by the benefits of increased detection of
malignancies would require a cost-benefit analysis incorporating long-term
data on the effect of earlier detection on morbidity.
View larger version (107K):
[in this window]
[in a new window]
[as a PowerPoint slide]
|
Fig. 3A —81-year-old woman underwent bilateral screening mammography.
Grade I ductal carcinoma in situ was seen at biopsy. Mediolateral oblique
(A) and craniocaudal (B) views show irregular mass that was
marked by radiologist (indicated by wax pencil mark) and missed by
computer-aided detection (CAD).
|
|
View larger version (84K):
[in this window]
[in a new window]
[as a PowerPoint slide]
|
Fig. 3B —81-year-old woman underwent bilateral screening mammography.
Grade I ductal carcinoma in situ was seen at biopsy. Mediolateral oblique
(A) and craniocaudal (B) views show irregular mass that was
marked by radiologist (indicated by wax pencil mark) and missed by
computer-aided detection (CAD).
|
|
View larger version (73K):
[in this window]
[in a new window]
[as a PowerPoint slide]
|
Fig. 3C —81-year-old woman underwent bilateral screening mammography.
Grade I ductal carcinoma in situ was seen at biopsy. Right craniocaudal
(C) and right mediolateral oblique (D) magnified views from
original screening examination show this mass (arrow) more
clearly.
|
|
View larger version (50K):
[in this window]
[in a new window]
[as a PowerPoint slide]
|
Fig. 3D —81-year-old woman underwent bilateral screening mammography.
Grade I ductal carcinoma in situ was seen at biopsy. Right craniocaudal
(C) and right mediolateral oblique (D) magnified views from
original screening examination show this mass (arrow) more
clearly.
|
|
The relatively high values obtained for both recall rate
[16] and cancer detection rate
[17] are a function of a
number of factors. Most notably, over the study period, substantial
improvements in patient positioning techniques by the radiologic technologists
were occurring. The images taken included substantially more breast tissue and
effectively served as baseline mammograms even for some patients who had
previously had screening mammograms. Further contributing to the effect was
that the study group did not represent a stable screening population, because
the number of patients presenting for baseline screening mammography at the
institution increased significantly over the course of the study period.
Our methodology reflected a prospective study in a clinical environment and
was largely concerned with the results of CAD as obtained from a real-life
practice rather than reproducible theoretic values. Although we recognize that
other investigators report a low reproducibility of marks with other CAD
systems when converting analog images to digital format
[18], we are not aware of this
issue being studied with the iCAD system, and our study did not address this
issue. In reality, however, analog-to-digital conversion is practiced by most
centers in this country using CAD technology; consequently, our study
represented the reality of clinical practice at this time.
The results of our study show that CAD decreased the number of
false-negatives in our study from 5 to 3, outside of the low end of the range
of potential benefit cited in a number of retrospective trials
[5,
7,
8,
19]. However, this
false-negative rate was determined in a clinical context and is not directly
comparable to values obtained from retrospective studies. In one of the
false-negative cases, CAD marked the lesion but the radiologist dismissed the
finding, and in the remaining two false-negative cases, neither revealed any
suspicious finding even on retrospective review. Thus, the results from
retrospective analyses may represent maximum values for the potential effect
of CAD on false-negative rates and are unlikely to be achieved in a clinical
setting.
View larger version (63K):
[in this window]
[in a new window]
[as a PowerPoint slide]
|
Fig. 4A —81-year-old woman underwent bilateral screening mammography.
Stage I invasive ductal carcinoma was seen at biopsy. Mediolateral oblique
(A) and craniocaudal (B) views show architectural distortion
that was marked by radiologist (indicated by wax pencil mark) and missed by
computer-aided detection (CAD).
|
|
View larger version (120K):
[in this window]
[in a new window]
[as a PowerPoint slide]
|
Fig. 4B —81-year-old woman underwent bilateral screening mammography.
Stage I invasive ductal carcinoma was seen at biopsy. Mediolateral oblique
(A) and craniocaudal (B) views show architectural distortion
that was marked by radiologist (indicated by wax pencil mark) and missed by
computer-aided detection (CAD).
|
|
View larger version (116K):
[in this window]
[in a new window]
[as a PowerPoint slide]
|
Fig. 4C —81-year-old woman underwent bilateral screening mammography.
Stage I invasive ductal carcinoma was seen at biopsy. Left craniocaudal
(C) and left mediolateral oblique (D) magnified views from
original screening examination show architectural distortion more clearly.
Arrow indicates location of lesion.
|
|
View larger version (105K):
[in this window]
[in a new window]
[as a PowerPoint slide]
|
Fig. 4D —81-year-old woman underwent bilateral screening mammography.
Stage I invasive ductal carcinoma was seen at biopsy. Left craniocaudal
(C) and left mediolateral oblique (D) magnified views from
original screening examination show architectural distortion more clearly.
Arrow indicates location of lesion.
|
|
By using the cancer registry for the hospital as an independent reference
standard, we were able to determine a sensitivity of 94% for a radiologist
with the assistance of CAD compared with 90% without the use of CAD. The
sensitivity of 79% measured for CAD alone is within the range of values
obtained in retrospective studies
[5,
7]. Our reported sensitivities
represent maximum values under the study design, because it is possible that
patients whose screening results were negative may have left the hospital
system and been diagnosed with cancer elsewhere, thus artificially lowering
the number of false-negatives documented. Although we did not report breast
density for the study population, a recent study by Brem et al.
[20] on the impact of breast
density in CAD for beast cancer reported no statistically significant effect
of breast density on overall CAD sensitivity.
It is worthwhile to note that the radiologists who participated in this
study are both experienced—one with 20 years of clinical experience, and
the other with fellowship training in breast imaging and 9 years of clinical
experience. Previous literature suggests that the use of CAD by an
inexperienced radiologist results in a greater increase in the cancer
detection rate and sensitivity than for an experienced radiologist
[21]. Thus, the values we
obtained for the effect of CAD on cancer detection rates and sensitivity of
screening may represent lower values on a range that would be expected in a
variety of clinical settings with radiologists of varying experience
levels.
Although it is convenient to look at the impact of CAD on the detection
rate, the ultimate goal of CAD is not to improve radiologic interpretation but
to improve treatment outcomes. In their 2001 study, Freer and Ulissey
[9] showed that use of CAD
increased the proportion of early-stage tumors detected (stage 0 or I) from
73% to 78%. Other recent studies show CAD's effectiveness in detecting small
lesions [22]. Our study
similarly shows an increase in proportion of early-stage tumors detected from
68% to 70%, with two of the three lesions detected by CAD, but not by the
radiologist, characterized as early stage, the third being a stage II tumor.
Although it has been established that CAD can lead to earlier detection of
malignant lesions, it is unclear whether this translates into the ultimate
goal of improved morbidity and mortality rates.
CAD has been suggested as a potential replacement to double reviewing on
the basis that its performance is comparable to that of double reviewing
[23]. Double reviewing in
screening mammography potentially increases the cancer detection rate by 3-15
women per 10,000 women screened and increases or decreases recall rates,
depending on the method of double screening used
[24-26].
Despite this improvement in sensitivity, the practice of double reviewing is
uncommon in the United States, owing largely to logistic, resource, and
financial considerations [14,
15,
27]. The results of this
study—showing a 4.7% increase in cancer detection rate with a
significant 2% increase in recall rate in absolute terms and no significant
change in positive predictive value—suggest that further research,
including economic analysis, is necessary to determine whether there is a role
for CAD in place of, or even in conjunction with, double reviewing
[16].
A number of studies have examined the effect of mammographic appearance and
tumor size on radiologist interpretation and prognosis
[7,
28] and on CAD sensitivity
[29-32].
Data from this study are consistent with these previous studies, showing a
greater sensitivity of CAD for overall microcalcifications than for masses and
architectural distortions. Specifically, CAD detected 100% (18/18) of
malignant lesions categorized as microcalcifications (Figs.
2A,
2B, and
2C) and 67% (20/30) of
malignant lesions categorized as masses (Figs.
3A,
3B,
3C,
3D,
4A,
4B,
4C, and
4D). Our study did not examine
how the system performed with different types of calcifications, which was the
focus of a recent study that used a different CAD system
[33].
It is noteworthy that CAD marked a malignant mass that the radiologist
dismissed, suggesting that the way that a radiologist uses CAD is shaped by
the radiologist's experience in using the technology and by the radiologist's
notions regarding CAD's strengths and weaknesses. Although CAD may hold
tremendous promise in improving the practice of screening mammography, its
efficacy is ultimately limited by the ability of a radiologist to interact
with CAD and capture that potential.
Fundamentally, an evaluation of CAD is a question of whether this
technology merely shifts the receiver operating characteristic curve of a
single radiologist or represents a point on a different curve entirely. The
debate over the relationship between recall rate and cancer detection rates is
especially relevant to this question
[34-36].
This article does not attempt to provide an answer; instead, it provides a
data point in a growing body of prospective data toward an assessment of the
use of CAD in the clinical setting. It would require the weight of studies
that use long-term cost-benefit analyses that look at effect on morbidity and
mortality to provide a truly definitive statement. Furthermore, CAD must be
shown to be a more cost-effective and feasible method for screening
mammography than alternatives, including increasing radiologist recall rates
or double-reviewer methodologies.
In summary, we show that the use of CAD in a clinical setting over a
26-month period increases the detection of cancer by at least 4.7% and the
sensitivity of mammography by at least 4%. These increases in cancer detection
and sensitivity come at the cost of a statistically significant 2% increase in
recall rate, but with no statistically significant impact on the positive
predictive value of biopsy. We report a "real" sensitivity of 94%
and specificity of 99% for the use of CAD in interpretation of screening
mammography.
References
- Fletcher SW, Elmore JG. Clinical practice: mammographic screening
for breast cancer. N Engl J Med 2003;348
: 1672-1680[Free Full Text]
- United States Preventive Services Task Force. Screening for breast
cancer: recommendations and rationale. Ann Intern Med2002; 137:344
-346[Abstract/Free Full Text]
- Bird RE, Wallace TW, Yankaskas BC. Analysis of cancers missed at
screening mammography. Radiology 1992;184
: 613-617[Abstract/Free Full Text]
- Goergen SK, Evans J, Cohen GP, MacMillan JH. Characteristics of
breast carcinomas missed by screening radiologists.
Radiology 1997;204
: 131-135[Abstract/Free Full Text]
- Warren Burhenne LJ, Wood SA, D'Orsi CJ, et al. Potential
contribution of computer-aided detection to the sensitivity of screening
mammography. Radiology 2000;215
: 554-562[Abstract/Free Full Text]
- Hendrick RE, Cutter GR, Berns EA, et al. Community-based
mammography practice: services, charges, and interpretation methods.
AJR 2005; 184:433
-438[Abstract/Free Full Text]
- Birdwell RL, Ikeda DM, O'Shaughnessy KF, Sickles EA. Mammographic
characteristics of 115 missed carcinomas later detected with screening
mammography and the potential utility of computer-aided detection.
Radiology 2000;219
: 192-202
- Brem RF, Baum J, Lechner M, et al. Improvement in sensitivity of
screening mammography with computer-aided detection: a multi-institutional
trial. AJR 2003;181
: 687-693[Abstract/Free Full Text]
- Freer TW, Ulissey MJ. Screening mammography with computer-aided
detection: prospective study of 12,860 patients in a community breast center.
Radiology 2001;220
: 781-786[Abstract/Free Full Text]
- Gur D, Sumkin JH, Rockette HE, et al. Changes in breast cancer
detection and mammography recall rates after the introduction of a
computer-aided detection system. J Natl Cancer Inst2004; 96:185
-190[Abstract/Free Full Text]
- Birdwell RL, Bandodkar P, Ikeda DM. Computer-aided detection with
screening mammography in a university hospital setting.
Radiology 2005;236
: 451-457[Abstract/Free Full Text]
- Baum JD, Raza S, Brem RF. Prospective evaluation of computer-aided
detection (CAD) in screening mammography: CAD impact on mammographic
interpretation time and work-up rate. (abstr)
Radiology 2001;221
: 521
- Guenin MA. How not to assess computer-aided detection for
mammography. (letter) AJR 2004;182
: 1599[Free Full Text]
- Elmore JG, Carney PA. Computer-aided detection of breast cancer:
has promise out-stripped performance? J Natl Cancer
Inst 2004; 96:162
-163[Free Full Text]
- Lerman C, Trock B, Rimer BK, et al. Psychological and behavioral
implications of abnormal mammograms and clinical breast examinations.
N Engl J Med 1998;338
: 1089-1096[Abstract/Free Full Text]
- U.S. Department of Health and Human Services, Agency for Health
Care Policy and Research. Clinical practice guideline number 13:
quality determinants of mammography. AHCPR pub. no. 95-0632.
Washington, DC: U.S. Department of Health and Human Services, Agency for
Health Care Policy and Research, 1994:78
-86
- Sickles E. Quality assurance: how to audit your own mammography
practice. Radiol Clin North Am 1992;30
: 265-275[Medline]
- Zheng B, Hardesty LA, Poller WR, Sumkin JH, Golla S. Mammography
with computer-aided detection: reproducibility assessment—initial
experience. Radiology 2003;228
: 58-62[Abstract/Free Full Text]
- Destounis SV, DiNitto P, Logan-Young WW, et al. Can computer-aided
detection with double reading of screening mammograms help decrease
falsenegative rate? Initial experience. Radiology2004; 232:578
-584[Abstract/Free Full Text]
- Brem RF, Hoffmeister JW, Rapelyea JA, et al. Impact of breast
density in computer-aided detection for breast cancer.
AJR 2005; 184:439
-444[Abstract/Free Full Text]
- Balleyguier C, Kinkel K, Fermanion J, et al. Computer-aided
diagnosis (CAD) in mammography: does it help the junior and the senior
radiologist? Eur J Radiol 2005;54
: 90-96[CrossRef][Medline]
- Brem RF, Hoffmeister JW, Zisman G, et al. A computer-aided
detection system for the evaluation of breast cancer by mammographic
appearance and lesion size. AJR 2005;184
: 893-896[Abstract/Free Full Text]
- Ciatto S, Rosselli Del Turco M, Burke P, et al. Comparison of
standard and double reading and computer-aided detection (CAD) of interval
cancers at prior negative screening mammograms: blind review. Br J
Cancer 2003; 89:1645
-1649[CrossRef][Medline]
- Dinnes J, Moss S, Melia J, et al. Effectiveness and
cost-effectiveness of double reading of mammograms in breast cancer screening:
findings of a systematic review. Breast2001; 10:455
-463[CrossRef][Medline]
- Kopans DB. Double reading. Radiol Clin North
Am 2000; 38:719
-724[CrossRef][Medline]
- Thurfjell EL, Lernevall A, Taube AS. Benefit of independent double
reading in a population-based mammography screening program.
Radiology 1994;191
: 241-244[Abstract/Free Full Text]
- Mitka M. Some radiologists want more money up front.
JAMA 2001; 285:281
-282[Free Full Text]
- Kallergi M, Carney GM, Galviria K. Evaluating the performance of
detection algorithms in digital mammography. Med Phys1999; 26:267
-275[CrossRef][Medline]
- Hadjiiski L, Chan HP, Sahiner B, et al. Improvement in
radiologists' characterization of malignant and benign breast masses on serial
mammograms with computer-aided diagnosis: an ROC study.
Radiology 2004;233
: 255-265[Abstract/Free Full Text]
- Karssemeijer N, Otten JD, Verbeek, AL, et al. Computer-aided
detection versus independent double reading of masses on mammograms.
Radiology 2003;227
: 192-200[Abstract/Free Full Text]
- Gur D, Stalder J, Hardesty L, et al. Computer-aided detection
performance in mammographic examination of masses: assessment.
Radiology 2004;233
: 418-423[Abstract/Free Full Text]
- Baker JA, Rosen EL, Lo JY, et al. Computer-aided detection (CAD) in
screening mammography: sensitivity of commercial CAD systems for detecting
architectural distortion. AJR 2003;181
: 1083-1088[Abstract/Free Full Text]
- Soo MS, Rosen EL, Xia JQ, et al. Computer-aided detection of
amorphous calcifications. AJR 2005;184
: 887-892[Abstract/Free Full Text]
- Gur D, Sumkin JH, Hardesty LA, et al. Recall and detection rates in
screening mammography. Cancer 2004;100
: 1590-1594[CrossRef][Medline]
- Brem RF. The never-ending controversies of screening mammography.
Cancer 2004; 100:1549
-1551[CrossRef][Medline]
- Yankaskas BC, Cleveland RJ, Schell MH, Kozar R. Association of
recall rates with sensitivity and positive predictive values of screening
mammography. AJR 2001;177
: 543-549[Abstract/Free Full Text]
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
This article has been cited by other articles:
|
|
|
|
 
M. Gromet
Comparison of Computer-Aided Detection to Double Reading of Screening Mammograms: Review of 231,221 Mammograms
Am. J. Roentgenol.,
April 1, 2008;
190(4):
854 - 859.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
R. F. Brem
Blinded Comparison of Computer-Aided Detection with Human Second Reading in Screening Mammography: The Importance of the Question and the Critical Numbers Game
Am. J. Roentgenol.,
November 1, 2007;
189(5):
1142 - 1144.
[Full Text]
[PDF]
|
|
|
|
|
|
|
R. F. Brem
Clinical Versus Research Approach to Breast Cancer Detection with CAD: Where Are We Now?
Am. J. Roentgenol.,
January 1, 2007;
188(1):
234 - 235.
[Full Text]
[PDF]
|
|
|
|
|
|
|
D. Gur and J. H. Sumkin
CAD in Screening Mammography.
Am. J. Roentgenol.,
December 1, 2006;
187(6):
1474 - 1474.
[Abstract]
[Full Text]
[PDF]
|
|
|
Top
Abstract
Introduction
