|
|
||||||||
Original Research |
1 Department of Radiology, Seoul National University College of Medicine, 28,
Yongon-dong, Chongno-gu, Seoul 110-744, Korea.
2 Institute of Radiation Medicine, Seoul National University College of
Medicine, Seoul, Korea.
3 Department of Medical Engineering, Seoul National University College of
Medicine, Seoul, Korea.
4 Department of Radiology, Stedelijk Ziekenhuis, Roeselare, Belgium.
Received October 10, 2006;
accepted after revision February 22, 2007.
Supported by grant 0412-MI00-0401-0007 from the Korea Health 21 R&D
Project, Ministry of Health & Welfare, Republic of Korea. 110-744,
Korea.
Abstract
|
|
|---|
SUBJECTS AND METHODS. The CTC data sets of 35 patients with at least one colonoscopically proven polyp were interpreted with a Hessian matrixbased CAD algorithm, which was designed to depict bloblike structures protruding into the lumen. Our gold standard was a combination of segmental unblinded optical colonoscopy and retrospective unblinded consensus review by two radiologists. Sensitivity of CAD for polyp detection was evaluated on both per-polyp and per-patient bases. The average number of false-positive detections was calculated, and the causes of false-positives and false-negatives were analyzed.
RESULTS. Ninety-four polyps were identified on colonoscopy.
Forty-six polyps were smaller than 6 mm and 48 were 6 mm or larger.
Seventy-five (79.8%) of these 94 polyps were identified by radiologists in a
retrospective review. When colonoscopy was used as a standard of reference,
the sensitivity of CAD was 77.1% for polyps 6 mm or larger. For large polyps
(
6 mm) that could be identified on retrospective review, the CAD
algorithm achieved sensitivities of 92.5% (37/40) and 91.7% (22/24),
respectively, on per-polyp and per-patient bases. There were an average of 5.5
false-positive detections per patient and 3.1 false-positive detections per
data set for CAD. The two most frequent causes of false-positives on CAD were
prominent or converging fold (78/191) and feces (50/191). Of the three polyps
6 mm or larger that were missed by CAD, two had a flat appearance on
colonoscopy and the remaining one was located in the narrow area between the
rectal tube and the rectal wall.
CONCLUSION. A Hessian matrixbased CAD algorithm for CTC has the potential to depict polyps larger than or equal to 6 mm with high sensitivity and an acceptable false-positive rate.
Keywords: colon computer-aided detection CT CT colonography neoplasm polyps
|
|
|---|
Computer-aided diagnosis (CAD) for CTC is attractive because it has the potential to circumvent these obstacles. Among the five typical tasks of the CAD system, the detection of polyp candidates requires an appropriate method to characterize these shape differences into polyps, folds, and the colonic wall. Several approaches have been proposed, including the use of the surface curvature with a rule-based filter, a volumetric shape index, and the extent of curvature [812]. And although each approach has been shown to be effective in detecting polyp candidates, one common problem with these approaches is that curvature-based polyp detection algorithms use only one or two parameters, thereby generating a myriad of false-positives. To overcome such shortcomings, new polyp detection algorithms using various parameters are being sought.
|
Therefore, the purpose of this preliminary study was to apply a new Hessian matrixbased CAD algorithm to CTC images and to analyze its performance on the basis of the reference standard of combined colonoscopy and retrospective unblinded review of CTC by experienced radiologists.
|
|
|---|
CT Colonography and Colonoscopy
CTC was performed using either of two MDCT scanners (LightSpeed Ultra, GE
Healthcare [n = 24]; or Sensation 16, Siemens Medical Solutions
[n = 11]). In all patients, colonic cleansing was performed with the
oral administration of 4 L of polyethylene glycol solution. All subjects
underwent CTC and successive same-day conventional colonoscopy within a 4-hour
interval. No spasmolytic agent was administered. For air insufflations, the
colon was gently insufflated with room air to maximum patient tolerance using
a mechanical inflator (Enema Teleflator CK-85, Kaigen), which is usually used
for double-contrast barium enemas, not for CTC examinations.
The scanning parameters used for the 8- and 16-MDCT scanners were 8 x 1.25 and 16 x 0.75 mm detector configuration, 1.5- and 1-mm slice thickness, 13.5 and 12 mm/s table feed, 0.8- and 0.5-second rotation time, respectively. Other parameters were 1-mm reconstruction interval, 512 x 512 matrix, 120 kVp, and 150 mAs for the prone and supine positions. The average scanning time to cover the entire colon was 1518 seconds for each position. For the second group (n = 11), supine images were acquired after the IV administration of 150 mL of iopromide at a rate of 3 mL/s after a 70-second delay. Contrast enhancement was performed to evaluate the extracolonic findings.
In all patients of the second group, conscious sedation was induced by IV sedatives (midazolam) during colonoscopy. Although all our practitioners would have liked to give midazolam liberally to all patients for colonoscopy, because of a lack of personnel and necessary resources only the second group received the sedative. All colonoscopic examinations were performed by one of five board-certified gastroenterologists using the segmental unblinding technique [6]. Polyp size was estimated by direct in vivo comparison with a 10-mm-long biopsy forceps along the axis of the base or head of the polyp, excluding the stalk if present. Flat lesions were defined as mucosal elevations with a height less than half the lesion diameter [15]. Except for some larger lesions, these flat polyps generally measure 3 mm or less in height. A histologic examination was performed on all lesions by one pathologist.
Overall Computer-Aided Polyp Detection Scheme
We transferred the CTC images to a PC and analyzed the images using the
prototype CAD scheme developed in our laboratory
(Fig. 1). The CT images were
interpolated along the transverse direction with a linear interpolation to
yield an isotropic volume data set (0.6 x 0.6 x 0.6 cm). Before
the detection of polyp candidates, colonic segmentation was done using the
seeded region-growing and isosurface-generation methods. A region-growing
threshold (475 H) for seeding the colonic lumen and a threshold
(800 H) for generating the isosurface were chosen. In region growing, a
seed point in the colonic lumen is manually selected, then all voxels
connected to the seed that have intensities less than a specified threshold
are identified by the computer. The surface of the lumen was then generated.
The surface of the colon was dilated and eroded by five voxels, and two images
were subtracted to extract a colonic wall of 10-voxel thickness. Then we used
a 3D shape-based filtering method based on the gradient vector and Hessian
matrix of volume intensity function to classify each structure into a polyp,
fold, or colonic wall, as suggested by Sato et al.
[13,
16].
|
Because we did not know the optimal setting in terms of gaussian smoothing factor and shape threshold, we investigated a variety of combinations using the two parameters. We designed three combinations that used the values of gaussian smoothing factor and shape threshold from low to high: gaussian smoothing factor and shape threshold of 1.5 (low) and 2.5 (low), of 1.5 (low) and 4.0 (intermediate), and of 2.0 (high) and 5.0 (high), respectively. We referred to these combinations as settings 1, 2, and 3, respectively. The three combinations were then analyzed using CTC data sets of 15 patients randomly selected from the 35 patients. A polyp was considered to be detected by the computer algorithm when one or more voxels considered to be a polyp candidate by CAD were in a polyp on CTC that matched the colonoscopy report.
To make this assessment, two radiologists evaluated the transverse and endoluminal 3D images. The sensitivity and total number of false-positive detections for each setting were determined for all 15 studies and compared using Fisher's exact test and the Friedman test. On the basis of these results, the optimum setting was identified. The optimal setting, as compared with the other settings, had both a high sensitivity and a low mean number of false-positives per patients.
Hessian matrix is the square matrix of second partial derivatives of a scalar-valued function and is well known for object recognition in computer vision and medical shape analysis. The details of the Hessian matrixbased method are presented in Appendix 1 and illustrated in Figure 2. All processes were performed automatically without human intervention.
Evaluation of CAD Results
The colon was divided into six segments: the cecum, ascending colon,
transverse colon, descending colon, sigmoid colon, and the rectum. For
evaluating CAD performance, we used two standards of reference: colonoscopic
results and results of retrospective and unblinded review. To establish the
ground truth, two radiologists (each with previous experience of 300 CTC
examinations) who were not blinded to the colonoscopic results,
retrospectively reviewed CTC images in a consensus manner. Lesions identified
by the radiologists were matched with the findings at colonoscopy on the basis
of the colonoscopic findings and the video registration. A polyp identified on
CTC was considered a true-positive if it matched the findings on colonoscopy
on the basis of colonic segment, size, shape, and anatomic interrelation to
haustral folds. For a given lesion to be considered a true-positive between
CTC and colonoscopy, it had to be in the same or an adjacent colonic segment,
the two recorded sizes had to be the same within a 30% margin of error, and
the lesion had to have similar morphologic features on both examinations. In
addition, the relationship of a lesion to thick colonic foldsthat is,
polyp on a fold versus polyp between foldswas also considered to
determine a true-positive match.
Sensitivity for polyp detection was computed as a function of polyp size and on per-polyp and per-patient bases. We computed sensitivity for polyp detection in two ways: first, by using the number of true-positive polyp detections that were made by the final results of colonoscopy after segmental unblinding to the CTC reports; and second, by using the number of true-positive detections made by the results of a retrospective and consensus review by two radiologists. The latter is especially useful for distinguishing the performance of CAD from the shortcomings of the CTC technique itself. For the latter, to avoid bias we did not use CAD to help locate the polyps not identified by radiologists. Instead, the radiologists reviewed the CTC images to define the causes of these 19 polyps that were not identified with even a retrospective analysis. Lesions were categorized by size on the basis of the colonoscopic results. Because we included only patients with at least one colonoscopically proven polyp, per-patient specificity was not calculated.
We reported the average number of false-positive detections for CAD rather than specificity. As is typical for other types of CAD used in radiology, CAD for colonic polyps usually resulted in one or more false-positive detections for each CTC examination (i.e., specificity approached 0).
In this study, two methods were used to calculate sensitivity and the average number of false-positive findings: first, evaluation based on volumetric data sets (by data set evaluation), and second, evaluation based on patients or cases (by patient evaluation). With both methods, the CAD scheme processed the supine and prone volumetric data sets independently to yield possible polyps. With the first method, supine and prone views of a patient were considered as different data sets, and the sensitivity per data set and the average number of false-positive findings per data set were calculated. With the second method, a polyp was recognized if it was detected on either the supine or the prone view of a patient, and the average number of false-positive findings per patient was calculated.
All the lesions depicted with CTC by CAD that were not seen at colonoscopy or did not match a colonoscopic finding were considered false-positive findings. All individual false-positives were carefully matched between data sets. The radiologists also inspected the cause of these false-positive findings in consensus using multiplanar reformatted views with reference to colonoscopic findings. Lesions that were missed on the CTC images by CAD were also reassessed retrospectively by the same two radiologists who had knowledge of the colonoscopic findings. A consensus judgment was made as to the cause of the false-negative results. Both the supine and prone views were used as needed during the retrospective interpretation.
|
|
|---|
|
In 15 patients who were used to find the optimal combination of the gaussian smoothing factor and shape threshold settings, a total of 26 polyps in 11 patients were identified at colonoscopy. Seventeen polyps were 5 mm or smaller in size, seven were 69 mm, and two were 10 mm or larger. Among them, five polyps of 5 mm or smaller were not identified in a retrospective review.
In 30 patients, thirty-five (76.1%) of 46 small polyps (
5 mm), 21
(75%) of 28 intermediate polyps (69 mm), and 19 (95%) of 20 large
polyps (
10 mm) were identified on the CTC images by two radiologists who
were not blinded to the results of colonoscopy. Thus, these sensitivities were
defined as the maximum possible sensitivities for the CAD algorithm. For the
19 polyps that were not identified even with a retrospective analysis, their
sizes ranged from 2 to 10 mm. Six were flat, 12 sessile, and one pedunculated.
Ten were adenomatous and nine were non-adenomatous polyps. Potential causes
for the false-negative diagnoses include that two polyps (10 and 8 mm) were
located in collapsed segments, one (7 mm, pedunculated) was buried in fluid or
stool, and one (7 mm) was obscured by the rectal balloon. No cause could be
identified for the remaining 15 false-negative diagnoses (three 7-mm, one
6-mm, and 12 < 5-mm polyps).
Diagnostic Performance of CAD on CTC
For 15 patients, the sensitivity and mean number of false-positives in the
three combinations of gaussian smoothing factor and shape threshold are
described in Table 2. Setting 1
had the highest sensitivity (42.9%) for all polyps and setting 3 had the
lowest sensitivity (23.8%). However, the differences of sensitivity among the
three settings were not statistically significant for all sizes. The average
number of false-positives was 14 in setting 1, 6.4 in setting 2, and 5.8 in
setting 3. Even though per-polyp sensitivity was highest in setting 1, the
average number of false-positives was unacceptably large (14 per patient). On
the contrary, because the per-polyp sensitivity of setting 2 for large polyps
(
6 mm) was the same as for setting 1 and the number of false-positives
(6.4 per patient) was significantly smaller than that of setting 1, we
determined that the optimal setting was setting 2, which was used for further
analysis of CAD performance for the testing set.
|
Table 3 shows the numbers of
true-positives and the sensitivities of CAD according to the size category for
the analysis on per-polyp basis. On the basis of the colonoscopic results, the
CAD algorithm using both supine and prone CTC data sets detected six small
polyps (
5 mm), 19 intermediate polyps (69 mm), and 18 large polyps
(
10 mm), for sensitivities of 13%, 67.9%, and 90%, respectively, and an
overall sensitivity of 45.7%. The sensitivity of CAD for 48 polyps 6 mm or
larger was 77.1% (37/48). When we used a combination of colonoscopy and
retrospective review as the reference standard, per-polyp sensitivities
increased to 17.1% for small polyps, 90.5% for intermediate polyps, and 94.7%
for large polyps. For the 40 polyps 6 mm or larger detected on retrospective
unblinded review by two radiologists, the CAD algorithm achieved a sensitivity
of 92.5% (37/40). Therefore, the overall potential sensitivity of CAD and
secondary interpretation by radiologists for polyps 6 mm or larger would be
83.3% (40/48). Of 43 polyps detected with CAD, eight were detected solely on
supine CTC images; 12, solely on prone CTC images; and 23, on both supine and
prone CTC images. All six cancers were detected by CAD, of which five were
detected on both supine and prone CTC images and one was detected on only the
prone images. Example CTC images of 20-, 9-, and 6-mm polyps detected by CAD
are shown in Figures 3A,
3B,
3C,
4A,
4B,
4C,
5A, and
5B.
|
|
|
|
|
|
|
|
|
Table 4 shows the results on a per-patient basis. Based on colonoscopic findings, per-patient sensitivity of CTC for polyps measuring 6 mm or more in diameter was 88% (22/25). For 24 patients having polyps 6 mm or larger detected on retrospective unblinded review, the sensitivity exceeded 91% (22/24).
|
Analysis of False-Negative and False-Positive Findings
Although both supine and prone CTC images were analyzed, a total of 51
polyps were missed by CAD: 25 adenomatous, 13 hyperplastic, and 13
inflammatory polyps. At a retrospective review of the CAD results, 19 missed
lesions were not identified by two radiologists: seven adenomatous, 10
hyperplastic, and two inflammatory polyps. Only three of the remaining 32
missed lesions were larger than 5 mm. The three retrospectively identifiable
polyps that were missed by CAD were all adenomatous polyps. Two of the three
polyps, 8 and 10 mm, were located in the sigmoid colon and had a flat
appearance on colonoscopy (Figs.
6A,
6B,
6C, and
6D). The remaining polyp was
located in the far distal rectum and was collapsed between the rectal wall and
rectal tube, resulting in its manifestation as a flat lesion even though the
original shape of the lesion was pedunculated (Figs.
6A,
6B,
6C, and
6D).
|
|
|
|
|
|
|
|
|
|
|
|
|---|
6 mm) and false-positive rate (28 lesions per patient) in the
detection of polyps at CTC in patients with known polyps
[812,
1822].
These data are clearly an indication that CAD for CTC needs further
improvement.
In our study, the Hessian matrixbased CAD algorithm provided
diagnostic performance comparable to previously developed CAD schemes. More
specifically, its per-polyp (92.5%) and per-patient (91.7%) sensitivities for
large polyps (
6 mm) were in the high range compared with values in the
published literature
[812,
1822].
However, although there is always a trade-off between sensitivity and the
false-positive rate in assessing the diagnostic performance of any CAD scheme,
our prototype CAD algorithm, for which we did not apply any method to reduce
the number of false-positives, showed a high rate of sensitivity for large
polyps (
6 mm) while maintaining a low false-positive rate (3.1 per data
set). The results of our study can be attributed mainly to the following
reason: Compared with other detection algorithms in which only one or two
shape parameters are used, the Hessian matrixbased algorithm uses three
morphologic structures (line, sheet, and blob) calculated from each voxel on
3D CTC images providing a detailed morphologic description of the candidate
surface.
The low gaussian smoothing factor and intermediate shape threshold we applied can also explain such positive results. Before applying the Hessian matrix decomposition to the CTC data set, gaussian blurring should be performed to enhance specific 3D local-intensity structures followed by the application of a shape threshold to further enhance detection capability for blobs (polypoid lesions) versus lines (folds in the colon) or sheetlike structures (colonic wall). The larger the applied gaussian smoothing factor is, the blurrier the colonic wall becomes. This means that the distinction between polyps, folds, and the colonic wall becomes weaker as a larger gaussian smoothing factor is applied to the images. This assumption was also proven in our study. Therefore, when we applied a low gaussian smoothing factor to yield a high true-positive rate (i.e., high sensitivity), the result was a higher false-positive rate (i.e., lower specificity) than we would have ideally wanted. However, we believed that an intermediate shape threshold somewhat compensated for the shortcomings of the low gaussian smoothing factor setting. Similar to the curvature used by Yoshida et al. [9] and Yoshida and Dachman [17], choosing a very high or very low shape threshold can lead to an increase in the number of false-positives, such as folds with high curvature and a colonic wall with low curvature. Even though the false-positive rate (3.1 per data set) of our study is in an acceptable range, there is room for further improvement, especially considering that we omitted false-positive reduction, which is usually performed as the last step of CTC CAD.
When applied to both supine and prone data sets, 11 of 48 large polyps
(
6 mm) were missed by CAD. However, only three of the 11 polyps could be
identified in a retrospective review by two radiologists. Two had a flat
appearance on colonoscopy, and the remaining polyp appeared as a flat lesion
on CTC although the original shape of the lesion was pedunculated. Our result
of poor CAD performance for flat polyps corresponds to that of previous
studies [22]. A lower
sensitivity of the colonic CAD system for flat polyps is well recognized.
Indeed, in a recent study regarding CTC CAD in a screening population, zero of
five flat adenomatous polyps 6 mm or larger were detected by CAD, whereas
48.5% of sessile polyps and 70.6% of pedunculated polyps of the same size were
detected by CAD [22]. The
difference in sensitivity as a function of shape was significant in that
study. Such limitations of CTC for the detection of flat polyps are also the
case with human reviewers as with CAD
[23].
The false-positive findings of our CAD system are mostly similar to those of other previously developed CAD algorithms [9, 24]. Most (139/191, 72.8%) false-positive detections in our study were caused by folds or retained feces. However, interestingly, 11% of the false-positive findings of our CAD scheme were caused by residual fluid in the colon. This rather high rate of false-positives caused by residual fluid can be explained by our use of wet preparation. Although wet preparation has the advantage of less feces in comparison with dry preparation, it retains a larger amount of fluid that shifts from side to side during the movement of the CT table, especially when the pitch parameter is high. Consequently, these gentle but perceptible waves simulate a blob, which is then identified by the CAD algorithm as a polyp. As a whole, we found that most false-positives can easily be identified as normal structures such as a colonic fold, fluid, or ileocecal valve. According to a recent article by Summers et al. [22], only a few (0.9%) of the false-positive CAD findings coincided with radiologists' false-positive findings. This suggests that most of the false-positive CAD findings would be determined by the radiologist to be unlikely to represent true polyps. Furthermore, preliminary evidence indicates that CAD's false-positives do not significantly impair radiologists' specificity even when almost 30 false-positives per patient are shown [25].
Our study has several limitations. First, we collected only a small number of polyps for this preliminary study; therefore, we are limited in making generalizations about the detection performance of our prototype CAD scheme. Evaluation of a larger number of polyps is desirable so that we may obtain a statistically solid performance measurement.
Second, we focused our CAD scheme only on the detection of polyp candidates, not on their characterization. The acronym "CAD" has been used to represent both computer-aided detection and computer-aided diagnosis. Initially, these might be considered identical, but an important difference exists when considering the number of blobs mimicking true polyps in the colon. Although our data show that it is possible to develop a CAD prototype that has a sensitivity comparable to those of radiologists and previously developed CAD algorithms with an acceptable number of false-positive detections, applying the final step of the CAD schemethat is, false-positive reductionto our prototype will be necessary to further decrease the number of false-positive detections.
Third, we used the same data set for both training and testing our CAD algorithm. This limitation is also related to the small number of cases. Although the validity of training and testing using the same data set, also known as cross-validation, is widely accepted in the medical and medical engineering fields, it could inflate the performance of CAD on the test set [26]. Accordingly, further prospective studies with a larger number of cases allowing adequate training and independent testing (external validation) are needed to better assess the performance of our CAD algorithm for detecting colonic polyps. If such a study produces encouraging results, the possibility of applying this method to clinical practice should be explored.
Finally, we performed this study to evaluate the feasibility of our CAD algorithm in detecting polyp candidates and to understand the types of false-positive findings generated using the Hessian matrixbased algorithm, and not to evaluate the benefit of our CAD scheme as a second opinion provider. To estimate the improvement of radiologists' detection based on our prototype CAD, an observer study using ROC analysis for radiologists of variable expertise with CTC should be conducted [27].
In conclusion, a Hessian matrixbased CAD algorithm for CTC has the potential to depict polyps larger than or equal to 6 mm with high sensitivity and an acceptable false-positive rate. Additional studies using a full CAD scheme that includes the final step of CADfalse-positive reductionare needed to further minimize the number of false-positive detections.
APPENDIX 1: The Hessian MatrixBased Method
|
|
|---|
![]() | (1) |
1,
2, and
3) and three Eigen vectors of each pixel. When an
object in the 3D image is approximated by a quadratic function, its shape is
characterized in terms of the Eigen values of the Hessian matrix. Eigen values and vectors were applied to basic conditions for each local structure (i.e., sheet, line, blob) and representative anatomic structures (i.e., colonic wall, fold, and polyp, respectively) to determine the likelihood of each shape. Figure 2 shows the Eigen values and Eigen vectors of each structure in 3D space. The lengths of the arrows in the figure indicate the magnitudes of the Eigen values. Figure 2 also shows schematic diagrams measuring similarity to local structures and the basic conditions of these local structures. Thus, in the line structure in general, the change of the gray level is large in the cross section and small along the direction of the line.
Sato et al. [13] proposed a
line emphasis filter, based on the above shape characterization, using the
Eigen values of the Hessian matrix. The Eigen values of the Hessian matrix are
the second-order derivatives of the change of the gray level along the
direction of the Eigen vectors. Letting the calculated Eigen values be
|
1|
|
2|
|
3|, the Eigen values
2 and
3, which
correspond to e2 and e3 along the directions of the
greater gray-level changes, are larger than Eigen value
1, which
corresponds to the direction e1 of smaller gray-level change.
Consequently, the following relationship applies to the interior of the line
structure:
![]() | (2) |
![]() | (3) |
|
|
|---|
This article has been cited by other articles:
![]() |
M. W. Lee, S. H. Kim, H. S. Park, J.-G. Lee, S. M. Joo, S. An, and B. I. Choi An Anthropomorphic Phantom Study of Computer-Aided Detection Performance for Polyp Detection on CT Colonography: A Comparison of Commercially and Academically Available Systems Am. J. Roentgenol., August 1, 2009; 193(2): 445 - 454. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |