Lung cancer is the leading cause of cancer-related death [1, 2], with the current fatality rate exceeding that of the next three most common cancers (breast, prostate, and colorectal) combined. Multiple studies have shown that low-dose screening helical CT scans can detect peripheral lung cancers at an early stage [2-4]. Because of the prevalence of benign, stable lung nodules in current and former smokers, once a nodule is detected, serial follow-up scans must be performed to detect growth. The introduction of MDCT scanners has led to lung cancer screening studies with a larger number of thinner slices, resulting in the detection of more nodules [5, 6]. This increase in the number of images per CT examination makes the process of CT interpretation more time consuming and tedious for the radiologist. This can lead to decreased detection sensitivity for nodules caused by reviewer fatigue [1, 7-9]. Therefore, computerized methods for nodule detection to assist the radiologist may become important. Recently, computer-aided detection (CAD) systems for lung nodules have been developed. Work has already begun to determine if CAD can serve as a stand-alone replacement for the radiologist screener, or, if not, can it serve a role as a “second pair of eyes” to supplement the radiologist screener [1, 7, 8, 10-12]. We evaluated a CAD system (ImageChecker CT LN-1000, R2 Technology) at our institution. The purpose of this study was to evaluate the value of this CAD system for automated pulmonary nodule detection and potential as a stand-alone or as a supplement to the radiologist.

There were 1,106 “suspected nodules” detected by either CAD or radiologists. After radiologist review, 628 were finally scored as true nodules. Of these, 518 (83%) of 628 nodules were detected by the radiologist and 456 (73%) of 628 were detected by CAD. Of those, there were 110 (17.5%) true nodules (Figs. 2A, 2B, and 2C), which would have been missed by the radiologist without using the CAD system. Importantly, six of the 150 (4%) patients were initially considered as “normal” by the radiologist and wouldn't have been recommended for follow-up without CAD. However, CAD missed 172 of 628 true nodules. In 56 (37.3%) of 150 patients, the results of nodule detection (number and location) corresponded between a radiologist and a CAD system; of these, 16 patients did not have any nodules. Finally, 478 (51.2%) of 934 of the “nodules” detected by CAD in 122 patients were rejected by the radiologist(s) as false-positives, for a rate of 3.19 (range, 0-26) false-positives per CT study.

An overview of the size distribution of true nodules and the corresponding detection performance of both techniques is given in Table 1.

Of the 115 small nodules (< 4 mm) missed by CAD (Table 1), besides the size algorithm limitation, lower density and contact to normal structures (e.g., pleura and vessel) further decreased the detection performance. Thirty-seven nodules had a peak attenuation value of less than -100 H; 34 were in contact with the pleura (Fig. 3A) or vessel (Fig. 3B).

Fifty moderate nodules (4 mm ≤ diameter ≤ 10 mm) were missed by CAD. Of these, six subpleural nodules were missed because they had pleural contact (Fig. 4A). Among 44 of 50 central and peripheral nodules without obvious pleural contact, 21 had lower density (CT_peak < -100 H) (Fig. 4B); 11 nodules were attached to a linear pleural tag or the normal intrapulmonary structures, such as fissure (Fig. 4C) and vessel (Fig. 4D) and thereby were excluded by the segmentation algorithm. There was no explanation for the other 12 nodules that were missed (Fig. 4E).

Seven large nodules (> 10 mm) were missed by CAD because of their continuity with normal structures, such as pleura (Fig. 5), fissure, or vessels.

The location of 628 true nodules and the detection performances of both techniques are presented in Table 2. We found the detection performance of the CAD system and the radiologist to be complementary (Fig. 6); the CAD sensitivity was higher in hilar (100%) and the central area (84%), and the radiologist's sensitivity was higher in the peripheral area (86%) and subpleural area (98%).

Of the 22 central nodules missed by CAD, six were less than 4 mm. Two of the six were attached to normal structures (Fig. 3B); the other four had low density (CT_peak < -100 H), which made them more difficult for CAD to identify. With the other 16 central nodules greater than 4 mm, CAD failed to detect them, possibly because of their lower density (n = 8; CT_peak < -100 H), or because they were abutting normal structures (n = 6) and were thereby excluded by the segmentation algorithm. No explanation was determined for the remaining two nodules.

Thirty-four subpleural nodules were missed by CAD presumably because of the segmentation algorithm. In addition, 27 of them were smaller than 4 mm (Fig. 3A). There were 116 peripheral nodules missed by CAD; 82 (71%) of 116 were smaller than 4 mm (moreover, 31 of 82 had a peak attenuation value < -100 H). Among the remaining 34 (29%) nodules greater than 4 mm, a CT_peak of less than -100 H was seen in 13 nodules (Fig. 4B). Attachment to the pleura (Fig. 5), vessel (Fig. 4D), and fissure (Fig. 4C) was seen in another 13 nodules; no obvious reason was found in the remaining eight nodules (Fig. 4E).

A CT_peak greater than or equal to -100 H was seen in 561 (89.3%) of 628 true pulmonary nodules, which is considered to represent a solid pulmonary nodule (Miller D et al., presented at the 2003 annual meeting of the Radiological Society of North America). The detection sensitivity of solid pulmonary nodules was 80% (448 of 561 nodules) with CAD and 81% for the radiologist (455 of 561 nodules). These are very close. Solid nodules were found only by CAD and missed by the radiologist in 106 (18.9%) of 561 nodules (Figs. 2A, 2B, and 2C). CAD missed 113 (20.1%) of 561 solid nodules. Seventy-eight (70%) of those were less than 4 mm (Fig. 3A). Among the remaining 35 (30%) of 113 nodules greater than or equal to 4 mm, attachment to the pleura (Figs. 4A and 5) or fissure (Fig. 4C) and vessels (Fig. 4D) was seen in 23 of 35; no obvious reason was found in the remaining 12 (Fig. 4E).

Reasons for the 478 false-positives from CAD and thus defined as false-positive (FP) include vessel, 262 (55%); pleural, 114 (24%); scars, 57 (12%); and others (e.g., bone, ground glass, and chest wall soft tissue), 45 (9%) (Figs. 7A and 7B).

CAD systems hold promise for helping radiologists to increase detection sensitivity. For detecting pulmonary nodules, sensitivity is increased by performing thinner slices with MDCT [8]. Our study shows that compared with a radiologist interpreting a reasonable number of medium-slice thickness (2.5-mm) images, there can be an increased detection of pulmonary nodules in the range of 21.2% by the addition of CAD using thinner slices (1.25 mm).

A wide variation exists in the detection sensitivity by CAD of pulmonary nodules in previously published studies [1, 7, 8, 10]. The detection sensitivity was 84% in a study by Armato et al. [1], who used CAD retrospectively in 31 patients with missed lung cancer. In Brown et al. [8], CAD detected 74% of nodules in 15 patients who had lung cancer. In a study by Wormanns et al. [7], only 38% of nodules were detected in 85 healthy subjects. Goo et al. [10] studied 50 volunteers and found a sensitivity of 65%. In our study, CAD had a 73% sensitivity in 150 patients.

False-positive rates range from three to 13 per CT study [1, 7, 8, 10]. In our study, there were 3.19 false-positives per study.

In our study, there was a high prevalence of nodules (mean, 4.18 per patient). This is because most cases were pulmonary nodule follow-ups of the original screening examinations performed in outside facilities, so very few cases had no nodules. Only 16 of the total 150 patients had no nodules detected, which might be a selection bias. Despite that, there were six patients whose nodules were only detected by CAD. In a screening situation, these patients would have been lost to follow-up, possibly missing cancers.

The high false-negative rate of CAD limits its application as a stand-alone technique. Missing true nodules by CAD in our study was predominately due to size limitation (≥ 4 mm), attenuation limitation (CT peak ≥ -100 H), and the segmentation algorithm (CAD only recognizes nodules entirely surrounded by lung parenchyma). Our results showed that these three factors are interactive in influencing the CAD detection performance. However, this CAD system still picked up 62 of the total 291 small nodules (< 4 mm) (Table 1). The reason is that although the current CAD algorithm targets nodules with diameters greater than 4 mm and less than 30 mm (in which a nodule is more typically described as a mass), the CAD algorithm also examines the nodular findings with diameters between 2 and 4 mm. For these smaller nodular findings, a stricter set of criteria on shape (i.e., more spherical) and location (i.e., clearly not part of adjacent structures) are applied to determine whether they are presented as CAD findings. Further development of the technology hopefully will overcome these deficits. Our results also showed that CAD and the radiologist worked in a complementary fashion in different lung zones because neither of them was able to find every nodule (Fig. 6). The radiologist has little difficulty in finding the peripheral and subpleural nodules even if they are small because there are no vessels of similar size near the pleural surface (Figs. 3A, 4A, 4B, 4C, 4D, and 4E). CAD is more sensitive in showing central nodules (Figs. 2A, 2B, and 2C), especially hilar nodules among the large vessels, which are prone to be misinterpreted as vessels and overlooked by the radiologist.

The high false-positive rate of CAD requires the radiologists to look at each suspected “nodule” to confirm its authenticity. In our study, the false-positive nodules were seen in 122 of the total 150 patients, which contribute to 3.19 false-positive rate per study. Although the analysis of nodules using one-way system (either by a CAD system or a radiologist) was satisfactory in 56 (37.3%) patients of our study, radiologists still needed to look at 46 of 56 CT scans to reject the false-positive. The causes of false-positives are vessel (54%), pleura (24%), and scar (12%). The others (10%) include consolidation, bone structure, and soft tissue of the chest wall. This has been described in other studies [1, 7, 8, 10, 12]. The particular software of our CAD system facilitates differentiating false-positives by the 3D-rendering image. The capability to rotate this image facilitates the distinction of a true nodule from the pulmonary vascular tree or pleural thickening. Although the false-positive rate is a drawback, CAD should have a high sensitivity even at the expense of a low specificity.

The main characteristic to diagnose malignant nodules is their growth over time [13-18]. Algorithms are already being developed to do temporal comparisons on follow-up studies. All previously detected nodules would be automatically assessed to see if they have increased in volume and at what rate. This feature probably will greatly enhance the diagnostic value of CAD systems in CT screening for early lung cancer [19].

CAD software is useful to supplement radiologists' detection performance. However, at present, it is not adequate as a stand-alone procedure. Furthermore, all suspected lesions detected by CAD must be interpreted by radiologists to rule out false-positives. In the future, temporal comparison should further improve the usefulness of CAD in the early detection of lung cancer.

We wish to express appreciation to John Mayo for help in reviewing the manuscript.

Address correspondence to P. L. Cooperberg.