| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Original Research |
1 Diagnostic Radiology Department, National Institutes of Health Clinical
Center, Bldg. 10, Rm. 1C368X MSC 1182, Bethesda, MD 20892-1182.
2 Department of Radiology, University of Wisconsin, Madison, WI.
Received October 26, 2007;
accepted after revision January 30, 2008.
Presented at the 2006 annual meeting of the Radiological Society of North
America, Chicago, IL.
Abstract
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
MATERIALS AND METHODS. CTC data were collected from the records of 104 patients undergoing screening for colorectal neoplasia. Most of the patients were at average risk, had CTC findings suggestive of polyps, and underwent colonoscopy. Patients underwent cathartic bowel preparation, were given an oral contrast agent, and underwent imaging in the prone and supine positions. The patients had 86 adenomas confirmed at same-day optical colonoscopy; 47 of these tumors were 10 mm in diameter or larger, and 39 measured 6–9 mm. The CTC data were analyzed with an existing CAD system for colonography that was trained with previously acquired data. In a previous non-polyp-enriched screening cohort, the standalone performance of the CAD system was 93.3% (28/30) sensitivity for adenomatous polyps 10 mm or larger, 51.1% (47/92) sensitivity for adenomas 6–9 mm, and a mean false-positive rate of 8.6 per patient. Sensitivity comparisons were made with findings in the previous study.
RESULTS. The CAD system had per-polyp sensitivities of 91.5% (43/47; 95% CI, 78.7–97.2%; p = 1.0) for adenomas 10 mm or larger and 82.1% (32/39; 65.9–91.9%; p = 0.0009) for adenomas 6–9 mm. The per-patient sensitivities were 97.6% (40/41; 85.6–99.9%; p = 0.6) for patients with adenomas 10 mm or larger and 82.4% (28/34; 64.8–92.6%; p = 0.047) for patients with adenomas 6–9 mm. The mean and median false-positive rates were 9.6 ± 9.6 and 7.0 per patient, respectively. Common reasons for CAD misses (false-negative findings) were the presence of adherent contrast medium, flat adenomas, and adenomas located on or adjacent to normal colonic folds. In a random sample, 72.5% (29/40) of false-positive findings were attributable to folds or residual feces.
CONCLUSION. The CAD system evaluated has a high level of performance in the detection of adenomatous polyps with CTC data from a polyp-enriched cohort different from that used to train the system.
Keywords: automated detection • colon • colon cancer • CT • image processing • virtual imaging
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Patient Population
The initial patient population of 182 consecutively registered screening
patients who had positive CTC findings was taken from a larger cohort
[6]. A positive CTC finding was
at least one polyp measuring 0.6 cm in diameter or larger. Patients underwent
CTC between April 2004 and September 2005.
The patients underwent same-day optical colonoscopy and polypectomy with pathologic analysis if CTC revealed at least one polyp 1 cm or larger. If the CTC finding was positive for polyps in the 6- to 9-mm size range, the patient was given the option of immediate optical colonoscopy or 1-year follow-up CTC. One hundred four (57.1%) of the 182 patients underwent immediate optical colono scopy for these indications. Twelve (11.5%) of the 104 patients had at least one false-positive CTC polyp finding. In two of these 12 patients, an adenoma also was found at CTC and confirmed at optical colonoscopy. The 104 patients who under went optical colonoscopy formed the study group because findings in these patients were confirmed or refuted. The demographic data for this cohort and the comparison (internal validation) cohort are shown in Table 1.
|
Five patients had a family history of colorectal cancer, and three had a history of irritable bowel syndrome. The other patients were at average risk even though subsequent filtering of negative cases revealed that most of them had a polyp. Five patients had symptoms related to the colon, such as abdominal pain and rectal bleeding. The other patients had no symptoms.
Bowel Preparation
Patients consumed a clear liquid diet for 24 hours before CT. For bowel
cleansing, the patients consumed one 45-mL dose of sodium phosphate
preparation (Fleet 1, Fleet Pharmaceuticals). For feces and fluid tagging, the
patients consumed one 250-mL dose of dilute barium sulfate (2.1% by weight,
Scan C, Lafayette Pharmaceuticals) and one 60-mL dose of diatrizoate sodium
(Gastro grafin, Bracco Diagnostics). These doses were one half of those used
in the internal validation study
[3].
CT
The colon was distended with patient-controlled insufflation of room air
(25%, 26/104, of patients) or automated CO2 delivery (75%, 78/104,
of patients) (ProtoCO2L, Bracco Diagnostics). CT was performed in
one breath-hold in each of the prone and supine positions. CT scanning
parameters are listed in Table
1.
Polyp Identification
Because hyperplastic polyps have little or no malignant potential, this
study was focused on adenomatous polyps. Eighty-six adenomas were confirmed at
optical colonoscopy. Thirty-nine of these adenomas measured 6–9 mm, and
47, including three adenocarcinomas and two villous adenomas, were 10 mm or
larger or larger. Among the polyps measuring 6–9 mm, the morphologic
features were 35 sessile, one pedunculated, and three flat. Among the polyps
10 mm or larger, the morphologic features were 20 sessile, 18 pedunculated,
six flat, one annular mass, and two unspecified. This count excluded four
adenomas (11, 7, 6, and 6 mm) not visible in retrospect on either the prone or
supine scans; these polyps are described in more detail later. The presence of
nine additional polyps (six 6–9 mm, four of which were found with CAD;
three 
10 mm, all three found with CAD) in six patients was confirmed with
optical colonoscopy, but no histopathologic diagnosis was recorded. These
polyps were not included in the polyp counts.
Polyp sizes for stratification of results were those determined at CTC. Polyp size measurements determined at CTC have been found to be more reliable and accurate than those measured at optical colonoscopy [7]. When a polyp was not found at CTC, optical colonoscopic size was used. Polyp size was determined before and independently of the process of CAD application.
The reference standard was established manually with the following method. Each polyp 6 mm or larger seen with optical colonoscopy was located on 2D and 3D prone and supine virtual colonoscopic images by an experienced radiologist using the V3D Colon software. To match on optical colonoscopy and CTC, polyps had to be located within the same or adjacent colonic segment and be within 50% of each another in size. With a graphical user interface, a voxel within each located polyp was marked manually to facilitate later review. The borders of a polyp were traced on each 2D CT slice containing the polyp. The markers and tracings were placed by trained research assistants under the supervision of an experienced radiologist. Polyps smaller than 6 mm were not incorporated in the reference standard and consequently were considered CAD false-positive findings if detected.
One of the two CTC scans (one prone, two supine) was missing for three
patients. Consequently, three CTC scans (supine or prone) were unavailable for
further analysis, leaving a total of 205 CTC scans of 104 patients. Four
adenomas (three 6–9 mm, one 10 mm) confirmed at optical colonoscopy
were not found in retrospect on at least one CTC scan (i.e., neither supine
nor prone). These adenomas were present in patients who had already been
referred for optical colonoscopic follow-up because of the presence of one or
more other CTC-identified polyps. One of these patients had multiple polyplike
bumps in different size categories in the same colonic segment, and it was not
feasible to determine which polyp was which. There were also three adenomas
found on only one of the two CTC scans (one 6–9 mm, two 10 mm). For
those identifiable in one view, the adenoma was marked and traced in that
view.
When there were multiple polyps of similar size (within 3 mm) within one colonic segment or adjacent segments and not all of these polyps were of the same histologic type, there was uncertainty about which was an adenoma confirmed at optical colonoscopy and which was a nonadenomatous polyp. These situations were resolved as well as possible with polyp size and morphologic features and appearance. Three patients had a total of eight polyps in which this ambiguity occurred.
False-Negative Findings by Radiologists
Five adenomas, all tubular, were missed by the radiologist at the original
prospective reading of the CT scan but were later found at optical
colonoscopy. These polyps were from patients who had other polyps seen at CTC
because only those patients went on to optical colonoscopy. Three of these
polyps (10, 12, and 20 mm) were visible in retrospect and were traced as
described earlier.
CAD System
The CAD system has been previously describ ed
[3,
8,
9]. It depicts the colonic
lumen and wall, electronically subtracts the contrast-enhanced colonic fluid,
calculates the colonic surface features, segments the potential polyps to
determine their 3D boundaries, and classifies the potential polyps as true or
false detections according to a set threshold. The system outputs the
locations of the polyp candidates on the CTC images. For purposes of improving
the display in the software, the polyp segmentations were dilated one voxel so
that the CAD marks were more easily visualized.
Matching of CAD of polyps with the reference standard was done in a completely automated manner without user interaction. If any voxel within a polyp candidate matched the voxels within a traced reference standard polyp, the polyp candidate was labeled a true-positive finding; otherwise, it was labeled a false-positive finding.
The CAD system contained a classifier for reducing the number of false-positive findings. For each polyp candidate, the classifier output a numeric score (classifier output score) of 0–1. Higher scores (near 1) indicated greater likelihood that the detection was a true polyp. In this CAD system a classifier was trained on polyps 8 mm and larger [3]. Compared with the previously used system [3], the CAD system had minor improvements, but the same classifier was used without additional training.
Assessment of False-Negative and False-Positive CAD Findings
The polyps missed with CAD were classified by an experienced radiologist
according to com mon reasons for detection problems for each scan (prone and
supine). The presence of oral contrast agent adhering to polyps was documented
[10]. Adherent oral contrast
medium can present as a thin coat ing or as one or more droplets on the
surface of polyps. In addition, a sample of 40 random false-positive CAD
findings were analyzed by the same radiologist and classified according to
common reasons for false-positive findings. At most, one false-posi tive
finding was sampled per patient. Knowledge of the causes of false-negative and
false-positive findings is helpful for improving the CAD system and guiding
physicians' use of CAD.
Statistical Analysis
The standard method for evaluating CAD performance is free-response
receiver operating characteristic (FROC) analysis. FROC analysis produces a
plot of the sensitivity of CAD for detecting polyps as a function of the
false-positive rate per patient. We also reported the sensitivity and
false-positive rate at a single operating point on the FROC curve that
corresponded to the same threshold on the classifier output score used in the
internal validation study. At this operating point the CAD software is
envisioned to be set for eventual clinical application. The operating point
was chosen in the previous study
[3] in a relatively flat part
of the FROC curve where there was diminishing gain in sensitivity as the
false-positive rate increased. For comparison, we computed and reported the
sensitivities of adenomatous polyp detection for the same threshold on the
classifier output score used in the internal validation study
[3]. The sensitivities and FROC
curves for the 6- to 9-mm category for the previous project were not reported
in that article because the classifier was trained on polyps 8 mm and larger.
For consistency we reanalyzed the CTC data from the previous project using the
same version of the CAD software used for this project. The 95% CIs for the
sensitivities at this operating point were calculated with an online
calculator [11]. To compare
the sensitivities of our results with the previous results
[3] at this operating point, we
used Fisher's exact test [12].
The sensitivity of detection of hyperplastic polyps also was reported. Values
of p < 0.05 were defined as statistically significant.
|
|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
|
|
|
|
|
The common reasons adenomas were missed with CAD (false-negative findings) are listed in Table 3. Most adenomas missed with CAD appeared flat or were located on or next to haustral folds. Adherent contrast medium was present in most (14/20, 70%) of the cases of false-negative adenoma. The mean and median false-positive rates were 9.6 ± 9.6 and 7 false-positive findings per patient (n = 104), respectively, compared with 8.6 ± 8.0 and 7 in the previous study [3]. The common reasons for false-positive CAD findings are shown in Table 4. Most of the false detections were related to normal colonic folds and residual feces, the latter often appearing as a plausible polyp.
|
|
Although hyperplastic polyps were not the principal aim of this study, 24 of 38 hyperplastic polyps 6 mm or larger were found at retrospective review. The sensitivities of CAD were 85.7% (12/14) and 70% (7/10) for hyperplastic polyps 6–9 mm and 10 mm or larger, respectively.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
10
mm) and medium (6–9 mm) size categories. The sensitivity in detection of
large polyps and the reasons for false-negative adenoma findings were
comparable with those in the CTC data used to develop the CAD system. All
three cases of cancer were detected. Two large adenomas missed at the original
prospective reading were detected; these adenomas were present in patients
with other polyps detected at the original prospective reading.
The sensitivity for 6- to 9-mm polyps was significantly greater than that in the previous study [3] (p = 0.0009 and p = 0.047 for peradenoma and per-patient sensitivities). Although the clinical relevance of such polyps is somewhat controversial, the increased sensitivity in this study is a worthwhile improvement because of the potential importance of polyps in this size range for guiding patient care. Because adenomas measuring 6–9 mm infrequently have pathologically important histologic features, such as villous architecture, high-grade dysplasia, and carcinoma [13, 14], some investigators [6] believe patients with such polyps can be placed into a surveillance population rather than undergo immediate polypectomy. Patients in the surveillance population would undergo more frequent CTC evaluation than those returning to the screening population. Another reason to detect polyps in this size range is to compensate for discrepancies in polyp size measurements between optical colonoscopy and CTC; polyp size occasionally is underestimated with CTC [15, 16].
The false-positive rates and the causes of false-positive and false-negative findings were comparable in the two data sets. In addition, we confirmed the finding from a previous study [10] that adenomatous polyps can be affected by adherent contrast medium. Adherence of contrast medium was commonly associated with false-negative results, suggesting that CAD systems may have to be specifically trained to recognize it.
In this study, the false-positive rate with the internal validation data set was higher than that reported previously [3] (8.6 vs 6.7 false-positive findings per patient). The difference likely occurred because the CAD system was not retrained after minor modifications were made in the CAD algorithm. Therefore, this false-positive rate is likely to be a conservative estimate of that potentially achievable after retraining.
The increased sensitivity for mediumsized polyps may be attributable to three differences in patient populations between this study and the earlier study [3]. First, the protocol for this study called for optical colonoscopy only after positive findings on CTC. Thus each patient in this study had at least one polyp visible prospectively on CTC that was of sufficient size that it triggered immediate optical colonoscopy. Consequently, these patients may have had polyps that were more conspicuous and perhaps also easier to detect with CAD. Although in this study the prospective sensitivity (40/42, 95%) of radiologists in detecting medium-sized adenomas confirmed at optical colonoscopy was greater than that (133/159, 83.6%) in a comparison study [2] in which all patients underwent optical colonoscopy regardless of CTC findings (p = 0.03, Fisher's exact test), the difference in sensitivity of CAD was much larger. Second, the bowel preparation and method of insufflation used in this study resulted in accumulation of less residual opacified colonic fluid and better distention [17]. With this CAD system, detection of polyps surrounded by air was much better than that of polyps under opacified fluid [3]. Consequently, more polyps in this study were likely to be surrounded by air and easier to detect with CAD. Third, the average slice thickness was smaller in this study. Thinner slices have been shown to be beneficial for polyp detection with CAD [18].
To be clinically useful, CAD must work as well on new CTC data sets as it does during the training and testing phases of CAD development. Such generalizability can be confirmed by evaluation of the software with CTC data from patients at an institution in a geographic locale different from that at which the original data set was collected. Confirmation of CAD performance with the new data indicates that the training and testing data incorporate the full range of variability of polyps in the population of interest. Such variability includes polyp size, location, shape, and CT attenuation. Confirmation also indicates that the range of normal variation was captured in the development, that is, training and testing, of data. These results indicate that the high performance of this CAD system is not idiosyncratic to the development data and that the system will probably perform well in broad clinical use in a screening population at average risk. It is not known whether this CAD system can be generalized to CTC data from patients of nationalities other than that of the original patients or to data from patients other than those at average risk.
This study is one of the largest standalone performance evaluations of a CTC CAD system. Other studies [19, 20] have been conducted with separate training and testing sets from one patient population and have achieved sensitivities and false-positive rates comparable with those we report. Other standalone performance trials [21, 22] of existing CAD systems with new data have been conducted. An external validation study [5] involved 25 patients drawn from a public repository and common to a small subset of the patients in our earlier study [3].
We did not evaluate the interaction of a radiologist with the CAD system. Results of such studies are beginning to appear [4, 23–28]. Early indications are that CAD may help the average radiologist interpret CTC images with sensitivity approaching that of experts but may also reduce specificity [26, 28]. In addition, radiologists sometimes ignore computer detections of true polyps [4]. The interaction between radiologist and CAD system requires further investigation.
This study had several limitations. Only patients with polyps of sufficient size at virtual colonoscopy to trigger immediate optical colonoscopy were included, potentially leading to enrichment with more conspicuous polyps. A study from our institution [29] showed that the incidence of positive screening results in the size ranges of interest found with CTC was comparable with the incidence of positive screening results at optical colonoscopy. In addition, in that study the numbers of advanced adenomas 10 mm or larger were equivalent in the primary CTC and primary optical colonoscopy patient groups. Although polyps discordant between CTC and optical colonoscopy would have been excluded from our cohort, such polyps are expected to be relatively few. We can estimate the correction factors from the per–adenomatous polyp sensitivities of CTC compared with segmentally unblinded optical colonoscopy for polyps 1 cm and larger (92.2%) and 6–9 mm (83.6%) using the data in one of our previous reports [2]. The actual correction factors would be higher in the current study because segmental unblinding was not used, leading to less correction (reduction) in the sensitivity of CAD. Although rigorous, segmentally unblinded colonoscopy is technically demanding and uncommonly used.
We did not account for intrapatient correlations in cases in which patients had more than one polyp. For a small number of patients, CTC scans and polyps were excluded because of missing data. For a small number of polyps, the histologic features were uncertain because similarly sized polyps with different histologic characteristics were colocated in the same or adjacent colonic segments.
We found that the performance of a CAD system in a polyp-enriched cohort
from a screening population at a medical center and geographic locale other
than those used for development of the system was similar to (for 10 mm
polyps) or significantly better than (for 6–9 mm polyps) the performance
of the system during development of the software.
Acknowledgments
We thank Andrew Dwyer for critical review of the manuscript. We thank J.
Richard Choi and William Schindler for supplying CTC data.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
