| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
1 All authors: Department of Radiology, Duke University Medical Center, Box 3808, Durham, NC 27710.
Received December 23, 2002;
accepted after revision April 22, 2003.
Address correspondence to J. A. Baker
(jay.baker{at}duke.edu).
Abstract
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
MATERIALS AND METHODS. Two commercially available CAD systems were used to evaluate screening mammograms obtained in 43 patients with 45 mammographically detected regions of architectural distortion. For each CAD system, we determined the sensitivity for revealing architectural distortion on at least one image of the two-view mammographic examination (case sensitivity) and for each individual mammogram (image sensitivity). Surgical biopsy results were available for each case of architectural distortion.
RESULTS. Architectural distortion was deemed present and actionable by a panel of expert breast imagers in 80 views of the 45 cases. One CAD system detected distortion in 22 of 45 cases of distortion (case sensitivity, 49%) and in 30 of 80 mammograms (image sensitivity, 38%); it displayed 0.7 false-positive marks per image. Another CAD system identified distortion in 15 of 45 cases (case sensitivity, 33%) and 17 of 80 mammograms (image sensitivity, 21%); it displayed 1.27 false-positive marks per image. Sensitivity for malignancy-caused distortion was similar to or lower than sensitivity for all causes of distortion.
CONCLUSION. Fewer than one half of the cases of architectural distortion were detected by the two most widely available CAD systems used for interpretations of screening mammograms. Considerable improvement in the sensitivity of CAD systems is needed for detecting this type of lesion. Practicing breast imagers who use CAD systems should remain vigilant for architectural distortion.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Although most breast cancers are identified on screening mammography as either a breast mass or a focus of microcalcifications, the third most common mammographic appearance of nonpalpable breast cancer is architectural distortion (i.e., a distortion of the parenchymal architecture without a concomitant mass) [7–9]. Because architectural distortion may mimic the normal appearance of overlapping breast tissue, this finding can be subtle and may be particularly difficult to detect [10]. Architectural distortion is a worrisome finding and has been reported to represent a breast malignancy in from one half to two thirds of the cases in which it is present [8, 9]. Because of its subtlety and potential for malignancy, architectural distortion is a common cause of false-negative findings on screening mammograms [11–13]. Therefore, evaluating the sensitivity of CAD systems for revealing this subtle appearance of breast cancer is valuable. We sought to determine the sensitivity of two commercially available CAD systems for identifying architectural distortion on routine screening mammograms.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Although each of the 51 cases was prospectively reported as architectural distortion, the rate of interobserver variability in choosing breast lesion descriptors [14] is high. Therefore, we had a panel of five breast radiologists, each of whom had at least 4 years of experience, determine the final morphology descriptor for each lesion in our study. These radiologists evaluated the routine craniocaudal and mediolateral oblique mammograms for the 51 cases. The panel members were unaware of the results of any additional imaging or the biopsy. For our study, a lesion was judged to be an architectural distortion if most (i.e., three of five) of the reviewers identified it as such on a particular mammogram. Each reviewer independently determined whether each lesion was best described as an architectural distortion without associated findings, a mass or focal density, or a non-actionable lesion because the lesion was not sufficiently conspicuous. This assessment was made for both the craniocaudal and mediolateral projections.
Of the 51 lesions described prospectively at the time of biopsy as architectural distortion, six were excluded because most of our panelists labeled the lesion as either a spiculated mass (five lesions) or a focal density (one lesion). The panel judged 45 lesions in 43 patients to be architectural distortions without associated findings; these 45 lesions composed our study population. One patient presented with bilateral synchronous foci of architectural distortion, and one patient presented with two foci of synchronous architectural distortion in the same breast.
CAD Analysis
Routine screening mammograms (craniocaudal and mediolateral) from each
patient were analyzed using two commercially available CAD systems:
ImageChecker M1000, version 2.5 (R2 Technology, Sunnyvale, CA) and SecondLook,
version 4.0 (CADx Medical Systems, Laval, QC, Canada). To create a digital
image of each mammographic view, we used the standard digitizer included with
each of the CAD systems (50-mm resolution for the R2 ImageChecker; 43.5-mm
resolution for the CADx SecondLook). The digital images were then analyzed
using proprietary software (included with the CAD systems) designed to
identify breast cancers presenting as microcalcifications or mass lesions.
Such systems identify masses by searching for a central density with radiating lines, suggesting spiculation, or for radiating lines without a central density, suggesting architectural distortion [4, 6, 15]. The output of the CAD system can be displayed on either two small video monitors or a single large flat-panel monitor or can be printed to paper, depending on the system and configuration. The CAD output shows suspicious foci of calcification and masses marked on a low-resolution reproduction of the mammograms. The R2 system uses small triangles to mark the location of possible calcifications and asterisks to mark possible masses; the CADx system uses rectangles to mark possible calcifications and ovals to mark possible masses.
Case Evaluation
The precise location of the architectural distortion was determined on both
the craniocaudal and mediolateral mammograms through consensus by two breast
radiologists with fellowship subspecialty training in breast imaging and at
least 4 years' experience. These radiologists had access to all diagnostic
images and wire localization images so that they could determine the actual
location of the lesion on the craniocaudal and mediolateral oblique
mammograms. They then compared this location with the location marked by the
CAD systems to determine whether the distortion on each mammogram was
correctly identified by each CAD system.
No universally accepted rule exists to determine whether a particular CAD mark is sufficiently close to a lesion to represent a true-positive mark [16]. Therefore, the two reviewing radiologists reached the decision of whether each lesion was correctly marked through consensus. When evaluating cases analyzed by the R2 ImageChecker—which places an asterisk at the site of a possible mass—we judged a CAD mark to be a true-positive if the asterisk was anywhere within the boundaries determined by the two radiologists as outlining the architectural distortion. In an effort to be fair, when evaluating CADx SecondLook—which places a variably sized oval to encompass the location of a possible mass—we judged an oval to be a true-positive mark if the center of the oval was anywhere within the boundaries outlining the distortion. Under a more lenient system, we might have allowed a portion—other than the center—of the oval that overlapped part of the architectural distortion to be counted as a true-positive mark; however, we did not encounter such a circumstance in any of the 45 lesions in our study.
The sensitivity of each CAD system for identifying architectural distortion in the 43 patients with 45 foci of architectural distortion was determined. The case sensitivity was determined by dividing the number of cases in which architectural distortion was correctly marked on either the craniocaudal or mediolateral mammogram by the total number of cases of architectural distortion. The image sensitivity was determined by dividing the number of mammograms on which architectural distortion was marked by the total number of projections on which it was visible (i.e., one or two mammograms per case). These sensitivities were calculated for both of the commercially available CAD systems we studied. In addition, the case sensitivity and image sensitivity for malignant lesions were determined for each CAD system. The McNemar test was used to compare the sensitivities of the two CAD systems.
Because there may be one or more false-positive marks per patient, specificity in CAD studies has historically been determined by the number of false-positive marks per image or per patient. For the purposes of our study, only marks that correctly indicated the location of architectural distortion were considered true-positive marks. Malignant lesions presenting as masses with appearances other than distortion or presenting as calcifications were excluded. The average number of false-positive marks per mammogram was determined for each CAD system, and statistical comparison of false-positive marks for the two systems was performed using the Student's paired t test.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
In all patients, the lesions were surgically excised. At the histopathologic examination, the focus of architectural distortion was found to be malignant in 27 (60%) of the 45 cases. Ten (22%) of the 45 cases represented invasive ductal carcinoma (Fig. 1A, 1B, 1C), 10 (22%) cases represented invasive ductal carcinoma with ductal carcinoma in situ (DCIS), and two cases (4%) represented invasive lobular carcinoma. DCIS was the sole finding in five (11%) of the 45 cases (Fig. 2A, 2B, 2C). The radiologist panel determined that architectural distortion was visible (and therefore actionable) on 51 images of the 27 malignant lesions. The lesion could not be identified on one of the two-view mammograms in three cases of malignancy, and the panel therefore deemed the lesion not actionable on that mammographic view.
|
|
|
|
|
|
Eighteen (40%) of the 45 cases of architectural distortion were benign. Twelve (27%) of the 45 cases represented a radial scar or a complex sclerosing lesion. The remaining six benign lesions (13%) included four cases of intralobular fibrosis, one case of benign proliferative change, and one case of a surgically confirmed postoperative scar.
The R2 ImageChecker system correctly marked at least one of the two screening mammographic views as containing a possible mass at the correct location in 22 of the 45 cases of architectural distortion (case sensitivity, 49%). The CADx SecondLook system correctly marked 15 cases of architectural distortion (case sensitivity, 33%). We found a trend toward better case sensitivity for the ImageChecker system for detection of both benign and malignant causes of architectural distortion, but the difference between the ImageChecker and the SecondLook systems did not reach the level of statistical significance (p = 0.10).
The ImageChecker system had virtually identical sensitivity for detecting malignant causes of architectural distortion as it did for detecting all cases of distortion. In malignant cases, the ImageChecker system successfully identified 13 (48%) of 27 cases of malignant distortion. The rate of detection for malignant cases for the ImageChecker system was significantly higher than that for the SecondLook system, which identified only five (19%) of the 27 malignant lesions (p = 0.027).
Architectural distortion without associated findings was seen on 80 mammographic views—35 cases in which it was visible in both craniocaudal and mediolateral views and 10 cases in which it was visible in only one of the two views. The focus of distortion was correctly identified in 30 of the 80 views (image sensitivity, 38%) by the ImageChecker system, significantly better than the 17 of 80 views correctly marked by the SecondLook system (image sensitivity, 21%) (p = 0.01).
The ImageChecker system was also significantly more successful at detecting malignant foci of distortion on each image (view) in which it was deemed actionable by the panel of radiologists. This system successfully detected the malignancy in 16 of the 51 images in which the distortion represented breast cancer (image sensitivity, 31%) compared with five of the 51 images (image sensitivity, 10%) for the SecondLook system (p = 0.01).
The two CAD systems marked different subsets of lesions (Fig. 1A, 1B, 1C) as possible malignancies. Of the 45 cases, nine cases (20%) were successfully identified on at least one view by both CAD systems. The ImageChecker system identified 13 cases (29%) of architectural distortion that were not identified by the SecondLook system. In comparison, the SecondLook system identified five cases (11%) that were not identified by the ImageChecker system. Eighteen (40%) of the 45 cases of architectural distortion were not detected by either CAD system (Fig. 2A, 2B, 2C).
Each CAD system also displayed a number of false-positive marks. On average, the ImageChecker system displayed 0.70 false-positive marks per image. This rate was statistically less than the 1.27 false-positive marks per image displayed by the SecondLook system (p < 0.0001).
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Architectural distortion has been described by the American College of Radiology in its Breast Imaging Reporting and Data System [18] as "[t]he normal architecture is distorted with no definite mass visible. This includes spiculations radiating from a point, and focal retraction or distortion of the edge of the parenchyma." The differential diagnosis of architectural distortion includes malignant lesions such as invasive ductal carcinoma, invasive lobular carcinoma, and DCIS and benign lesions such as a surgical scar, a radial scar, a complex sclerosing lesion, fat necrosis, and intralobular fibrosis [10]. Most malignancies presenting as architectural distortion are invasive rather than in situ carcinomas. In one study of breast malignancies, two thirds (16/24) of the cases presenting as architectural distortion were invasive ductal carcinoma, 21% (5/24) were invasive lobular carcinoma, and 13% (3/24) were DCIS alone [19]. These figures parallel our study results, with invasive ductal and lobular carcinomas accounting for 81% (22/27) of the malignancies.
Although several benign entities are included in the differential diagnosis, the malignancy rates for architectural distortion range from almost one half to two thirds of the cases [8, 9]. Unfortunately, mammography cannot be used to differentiate benign from malignant foci of architectural distortions. Several studies have documented that mammographic features such as the length of radiating lines or the presence of a central density cannot be used to differentiate benign from malignant lesions [10, 20–22]. Because architectural distortion is frequently caused by malignancy and because benign causes of distortion cannot be excluded on the basis of imaging features, identification of all cases of distortion is essential, as is performance of a definitive biopsy.
Architectural distortion has been reported to be the third most frequent mammographic appearance of breast cancer [7, 8], but distortion may be challenging to detect. In one study of malignancies overlooked by radiologists on screening mammograms, architectural distortion accounted for nine (12%) of the 77 missed cases [13]. Another study placed the percentage of missed malignancies presenting as architectural distortion as high as 45% [11].
Despite the subtlety and potential for malignancy of architectural distortion, few reports investigating the efficacy of CAD algorithms have specifically addressed detection of distortion. In one study, Evans et al. [4] investigated CAD sensitivity for the detection of lobular carcinoma and found that 17 (85%) of 20 cases of lobular carcinoma presenting as architectural distortion were successfully marked by a CAD system. Likewise, Birdwell et al. [1] evaluated 115 breast cancers overlooked by the interpreting radiologist and found that five (83%) of six missed breast cancers presenting as architectural distortion were successfully identified by one CAD system.
However, our results indicated that the two most widely available commercial CAD systems had only limited success in detecting architectural distortion. The more successful system in our study identified distortion on at least one mammographic view in slightly fewer than half of our cases, whereas the other system identified just one third of cases. Compared with their success in identifying calcifications and masses, the success of these systems was substantially less in identifying architectural distortion. Of the 80 mammographic views on which a distortion was deemed visible and actionable by most members of the radiologist panel, only one case in five (on average) was identified by one of the CAD systems, with the more successful system performing only moderately better than the less successful system. Particularly troubling is the finding that the sensitivity of both systems was either the same or moderately worse in detecting distortions caused by malignancy as in detecting distortions due to benign causes such as radial scars.
Our study is limited in that it only examined the sensitivity of CAD systems for those foci of architectural distortion initially detected by a human observer. Theoretically, a CAD system can detect other foci of architectural distortion that may have been overlooked by a human observer until prompted to further evaluate the area by a CAD mark. Therefore, a CAD system could actually detect a higher percentage of architectural distortion lesions than we found in our study if the cases that were only identified by a CAD system were included. Future research on the sensitivity of CAD systems to specific mammographic features could include a large trial with prospective descriptions of the morphology of all lesions detected by a human observer or a CAD system.
Although CAD systems are largely successful at identifying breast masses and microcalcifications, the finding that they are less successful at identifying architectural distortion is not surprising, given the techniques used in the CAD analysis of mammograms. Many algorithms for breast mass detection rely heavily on the presence of a central density [4, 5, 23–30]. Techniques such as template matching [24] and low-pass or band-pass filtering (e.g., gaussian filtering) [30, 31] function by searching for a region that is relatively more dense than the surrounding tissue; left-to-right subtractions search for asymmetric density [27, 28]. Such algorithms are not designed to recognize the radiating lines that define architectural distortion.
Other experimental CAD algorithms may identify radiating lines. Examples include radial-edge gradient-based algorithms [32, 33], edge profile acuteness (i.e., sharpness) measurements [34], and rubber band straightening transform analysis [35] for evaluating mass edge features. Such techniques have the potential to be more sensitive in detecting architectural distortion, regardless of the presence of a central density.
Because the techniques used by commercial vendors are proprietary, we cannot determine which mass-detection algorithms each system uses. Prior reports have indicated that the ImageChecker M1000 software searches for features common to malignant masses such as areas with central density and radiating lines [4]. The fact that the CAD system used by Evans et al. [4] had lower sensitivity for detecting architectural distortion than for detecting other lesions may be explained in part by the researchers' statement that "[w]hen no central density is found, the radiating lines must be more pronounced to be marked." This approach limits the number of false-positive marks for each case because normal overlapping tissue (e.g., Cooper's ligaments) can mimic the radiating lines of architectural distortion, deceiving both a radiologist and a CAD algorithm. Each of the other commercial systems approved by the United States Food and Drug Administration, including SecondLook and the more recently approved MammoReader (iCAD, Nashua, NH), must make similar trade-offs between sensitivity and false-positive marks.
In our study, the more sensitive of the two systems (ImageChecker) also had a significantly lower number of false-positive marks per image. Systems that generate many false-positive marks may result in a true-positive mark being ignored by a radiologist over-whelmed by distracting prompts. Therefore, the false-positive rate of a CAD system must be considered along with its sensitivity.
The purpose of our investigation was to test the sensitivity of increasingly available CAD systems to determine whether such systems are as successful in detecting worrisome foci of architectural distortion as they are in detecting more common breast masses and calcifications. Although one of the CAD systems was significantly more sensitive than the other for detecting architectural distortion, a study by Nelson et al. [36] found that the three commercially available mammography CAD systems—the R2 ImageChecker, the CADx SecondLook, and the iCAD MammoReader— all performed with nearly identical sensitivity in a study of 128 malignant masses and clusters of calcifications. We found that both systems had substantially lower rates for identifying architectural distortion than the previously reported rates of those systems for detecting more common masses and calcifications. Clearly, both systems need to be improved, given that one half to two thirds of the cases of architectural distortion were not identified by the two most widely available commercial CAD systems.
Because of the similarity between architectural distortion and overlapping fibroglandular tissue, improvement in detection may prove difficult without a concomitant—and perhaps unacceptable—increase in the number of false-positive marks per image. Nevertheless, now that CAD systems can successfully identify almost all malignant calcifications and most malignant masses, the capability of the systems to detect the more subtle signs of malignancy, such as architectural distortion, should be addressed. In fact, despite the introduction of computer systems to assist radiologists in the challenging task of identifying breast cancers, little has changed since Sickles' landmark study of 300 consecutive cases of nonpalpable breast cancer [7] in which he concluded that "[t]o take advantage of mammography, [radiologists] must search diligently not only for characteristic tumor masses and clustered calcifications, but especially for more subtle signs of malignancy" (e.g., architectural distortion). Our results have shown that further work is necessary to develop CAD systems that are more capable of assisting radiologists in that diligent search.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
This article has been cited by other articles:
|
|
 |
|
  D. Georgian-Smith, R. H. Moore, E. Halpern, E. D. Yeh, E. A. Rafferty, H. A. D'Alessandro, M. Staffa, D. A. Hall, K. A. McCarthy, and D. B. Kopans Blinded Comparison of Computer-Aided Detection with Human Second Reading in Screening Mammography Am. J. Roentgenol., November 1, 2007; 189(5): 1135 - 1141. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R. L. Ellis, A. A. Meade, M. A. Mathiason, K. M. Willison, and W. Logan-Young Evaluation of Computer-aided Detection Systems in the Detection of Small Invasive Breast Carcinoma Radiology, October 1, 2007; 245(1): 88 - 94. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. E. Baker, L. Bogoni, N. A. Obuchowski, C. Dass, R. M. Kendzierski, E. M. Remer, D. M. Einstein, P. Cathier, A. Jerebko, S. Lakare, et al. Computer-aided Detection of Colorectal Polyps: Can It Improve Sensitivity of Less-Experienced Readers? Preliminary Findings Radiology, October 1, 2007; 245(1): 140 - 149. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
P. Skaane, A. Kshirsagar, S. Stapleton, K. Young, and R. A. Castellino Effect of Computer-Aided Detection on Independent Double Reading of Paired Screen-Film and Full-Field Digital Screening Mammograms Am. J. Roentgenol., February 1, 2007; 188(2): 377 - 384. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. H. Taplin, C. M. Rutter, and C. D. Lehman Testing the effect of computer-assisted detection on interpretive performance in screening mammography. Am. J. Roentgenol., December 1, 2006; 187(6): 1475 - 1482. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. M. Ko, M. J. Nicholas, J. B. Mendel, and P. J. Slanetz Prospective assessment of computer-aided detection in interpretation of screening mammography. Am. J. Roentgenol., December 1, 2006; 187(6): 1483 - 1491. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
V. R. Pai, N. E. Gregory, A. E. Swinford, and M. Rebner Ductal Carcinoma in Situ: Computer-aided Detection in Screening Mammography Radiology, December 1, 2006; 241(3): 689 - 694. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. C. Alexander, B. C. Yankaskas, and K. W. Biesemier Association of stellate mammographic pattern with survival in small invasive breast tumors. Am. J. Roentgenol., July 1, 2006; 187(1): 29 - 37. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. J. Morton, D. H. Whaley, K. R. Brandt, and K. K. Amrami Screening Mammograms: Interpretation with Computer-aided Detection--Prospective Evaluation Radiology, May 1, 2006; 239(2): 375 - 383. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. J. Brenner, M. J. Ulissey, and R. M. Wilt Computer-Aided Detection as Evidence in the Courtroom: Potential Implications of an Appellate Court's Ruling Am. J. Roentgenol., January 1, 2006; 186(1): 48 - 51. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C. V. Ongeval, H. Bosmans, and A. Van Steen Current challenges of full field digital mammography Radiat Prot Dosimetry, December 1, 2005; 117(1-3): 148 - 153. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. S. Soo, E. L. Rosen, J. Q. Xia, S. Ghate, and J. A. Baker Computer-Aided Detection of Amorphous Calcifications Am. J. Roentgenol., March 1, 2005; 184(3): 887 - 892. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. Gur, J. S. Stalder, L. A. Hardesty, B. Zheng, J. H. Sumkin, D. M. Chough, B. E. Shindel, and H. E. Rockette Computer-aided Detection Performance in Mammographic Examination of Masses: Assessment Radiology, November 1, 2004; 233(2): 418 - 423. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. A. Baker, J. Y. Lo, D. M. Delong, and C. E. Floyd Computer-aided Detection in Screening Mammography: Variability in Cues Radiology, November 1, 2004; 233(2): 411 - 417. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCR |
