Raw data from thoracic CT scans of 50 consecutively registered patients referred for cancer staging, lung nodule screening, or follow-up were prospectively collected between January and March 2009. The images were read by a consensus panel of two thoracic radiologists with 9 and 10 years of experience. Lung nodules were found in 41 patients (22 men, 19 women; median age, 55.5 years; range, 35–83 years; median weight, 73 kg; range, 42–120 kg), and a total of 125 nodules were selected. The nine patients with normal (true-negative) thoracic CT findings served as a control group. The study protocol was approved by the local institutional review board, and informed consent was waived because of the retrospective data processing.

All image data were obtained with a 64-MDCT scanner (Somatom Sensation, Siemens Healthcare) with collimation of 64 × 0.75 mm, pitch of 2, 120 kVp, and reference tube current–time setting of 300 mAs. Automated tube current–time modulation (CARE Dose, Siemens Healthcare) was engaged to ensure a similar noise index for each patient. Acquisition parameters were identical to the parameters used in our standard clinical protocol. All patients were scanned in the supine position from the lung apex to the base of the chest at breath-hold full inspiration. No IV contrast material was administered.

After reconstruction of the original DICOM images with an image matrix of 512 × 512 and a B45 kernel, 12 additional image sets were reconstructed from the same raw data simulating different dose levels. Computer-calculated noise was superimposed on the original raw dataset by use of the validated CT software (Somatom Noise+ V5.Obeta, 2006, Siemens Healthcare) [21], and reference tube current–time settings of 220, 180, 140, 100, 80, 60, 50, 40, 30, 20, 10, and 5 mAs were simulated. This process resulted in 13 datasets per patient available for image analysis. The rather low kernel was a compromise for improving the 3D reformation and the accuracy of semiautomated volume measurement.

A consensus panel of two thoracic radiologists (9 and 10 years of experience) identified small nodules (≤ 8 mm), making sure that approximately one half of the selected nodules were solid (maximum attenuation, ≥ –300 HU) and one half ground-glass nodules (< –300 HU). Cuboidal regions of interest (3 × 3 × 1.5 cm volumes) containing only one nodule each were selected by transferring the center coordinates to the lung nodule evaluation platform (Internet readout workstation described later). To compensate for potential enhanced nodule detection for limited FOV and z-axis, only smaller (≤ 8 mm) nodules were selected. A total of 125 cuboids showing a single nodule were selected for subsequent image analysis. A total of 27 cuboids of the nine patients without disease were randomly collected and added for image analysis. The greatest axial dimension of each CT finding was measured with digital calipers, and its maximum attenuation was noted in Hounsfield units.

Detectability of nodules—Interpretation was performed with a lung nodule evaluation platform that was not commercially available. The system featured a split screen for automated display of two orthogonal cross sections (axial and coronal planes combined for multiplanar reconstruction) in stacked cine mode of a cuboidal region of interest. For our study the screen was zoomed to the axial cross sections, and the coronal plane was hidden. Before appearing on the screen, the cuboids were randomly axially rotated to positions 0°, 90°, 180°, 270°. The cuboids then could be browsed slice by slice. In a separate integrated window, readers recorded their answers by mouse click. Data then were automatically transmitted online to a statistician. The lung nodule evaluation platform was installed on a PC workstation (Dell E520, 2.1 GHz, Intel Core 2 Duo, 4 GB RAM) with a 20-inch (51 cm) liquid crystal display (Dell Ultrasharp, 1600 × 1200 pixels). Ambient light conditions, which were similar to our clinical reading conditions, and the display window settings (window center, –600 HU; width, 1500 HU) were kept constant for all reading sessions and readers.

Size of nodules—Digitized measurements of nodule size were performed with a Mac Book Pro 15-inch (38 cm) screen (Apple Computer) running OsiriX Imaging Software (OsiriX). Center coordinates were used to find the nodules in the full CT datasets. Zoom of 2000% was used for all nodules. Computer-aided volume measurement (LMS Lung/Track, Median Technologies) was performed. This semiautomatic lesion measure ment solution seg ments pulmonary nodules identi fied by radiologists (radiologist chooses nodule by mouse click). The segmentation is the result of a region-growing algorithm followed by thresholding on attenuation (Hounsfield units) and concluded by mathematic morphologic operations (erosion and dilation) to separate the suspicious lesion from surrounding anatomic structures. The method published by Bland and Altman for assessing agree ment between two methods of clinical measure ment [22] was used to determine intraobserver and inter observer variability and agreement of volume measurements at different CT tube current–time levels. This method was used to calculate limits of agreement.

Detectability of nodules—Three readers (board-certified radiologists who had 5 years [reader 1], 6 years [reader 2], and 12 years [reader 3] of experience in interpretation of chest CT scans) were instructed to identify lung nodules. A pilot study with a different reader (9 years of chest CT experience) of 36 cuboids from 20 patients at five dose levels (300, 100, 50, 20, and 5 mAs) was performed to decrease the number of readout cases. Only at dose levels of 20 and 5 mAs were different answers given compared with 300 mAs. Therefore, we limited the evaluation to seven relevant tube current–time levels: 300, 100, 40, 30, 20, 10, and 5 mAs. The readers' answers were recorded by use of the lung nodule evaluation platform. Image interpretation took place in multiple reading sessions with blocks not exceeding 1 hour of successive reading to minimize reader fatigue. Presentation of the cases to the readers was randomized, and the cuboids were randomly rotated over 360° in the axial plane to minimize recognition bias.

Size of nodules—Comparison of measurements on two consecutive scans requires knowledge of observer agreement because all observed differences between measurements of the same nodule on different scans can be attributed to intraobserver variability or to interscan variability. Two readers (other than the aforementioned board-certified radiologists who had 3 years [reader 4] and 6 years [reader 5] of experience in interpretation of chest CT) measured the size and volume of the 59 solid nodules at all 13 tube current–time levels manually and with the computer-aided lesion measurement solution software. The readers measured volume by multiplying the longest axial axis (according to the Response Evaluation Criteria in Solid Tumors [RECIST]), the perpendicular short axial axis, the longest coronal z-axis, and π/6. Reader 4 performed the same evaluation 3 months later to determine intrareader agreement. The computer measurements were obtained with segmentation and threshold methods for automated volume and longest axial diameter measurement (RECIST). This semiautomatic process was performed by both readers. Reader 5 repeated the measurement with a delay of 1 day for intrareader agreement. This technique was chosen because it was our hypothesis that repeated measurement with different starting points might give different volumes.

Detectability of nodules—Individual results from the three readers were compared with the standard of reference. Sensitivity and specificity for each reader in detection of pulmonary nodules were calculated at each dose level. Paired comparisons of the sensitivity of detection at each lower dose level against the corresponding detection at 300 mA were performed. The null hypothesis of equality of sensitivity for this dose pair was then tested by McNemar test for each reader [23].

For comparison of the effect of tube current–time level and nodule size and density on sensitivity for lung nodules the odds ratio for each variable and reader was calculated. For practical reasons, data on each variable were dichotomized: size 3 mm and larger versus less than 3 mm, attenuation –300 HU versus less than –300 HU (ground-glass opacity), and standard dose (300 mAs) versus low dose (5 mAs). Odds ratios and sensitivities were calculated, and p values for each variable were analyzed by chi-square test. If an absolute cell frequency was lower than 5, the Fisher exact probability test was applied. A value of p < 0.05 was considered to indicate a statistically significant difference. To rule out a bias of dichotomizing level setting and to obtain a more general statement, the following logistic regression model was fitted to diagnosis [24]:where p is the probability of a reader's giving a positive diagnosis (sensitivity); c is the intercept; b₀, b₁, and b₂ are regression coefficients (log odds ratios); LD is log(dose) in milliampere-seconds (tube current–time level); st indicates standardized, size is the maximum axial diameter of a nodule (average of readers 4 and 5 at standard tube current–time level); HU is attenuation; and error is the specification error. The variables were standardized, and the mean was made 0 with 1 SD. The advantage of using this calculation is that the regression coefficients b₀, b₁, and b₂ become comparable on the same scale. Because a logarithmic relation between dose and sensitivity can be assumed, the logarithmic dose was used.

Size of nodules—The method described by Bland and Altman [22] for assessing agreement between two methods of clinical measurement was used to determine intraobserver and interobserver variability and agreement of volume measurements between the standard tube current–time setting of 300 mAs and all lower tube current–time levels (220–5 mAs). This method of calculation of limits of agreement results in the range of 95% of all volume measurement errors. Because the percentage of changes is more important than absolute volume values, the volume measurement error (VME) is given as a percentage of absolute volume measurement difference (V₁ – V₂) in relation to nodule volume, where V₁ is the volume of the first reading. The true nodule volume is not known; therefore, the mean nodule volume of both measurements (V₁ + V₂) / 2 is the best estimate obtainable [22]:

In addition, the 95% CI of the mean volume measurement error of the upper and lower limits of agreement was calculated. The limit of agreement was calculated from the SD of the measurement differences (VME), and the 95% CI was derived from the standard error for normally distributed data, which can be assumed for measurement differences [22] (Fig. 1). To find significant differences in volume measurements, the 95% CI of the volume measurement error should not include 0 for significant systematic bias. The corrected SD of measurement differences was applied for repeated measurements [22].

Diameter and volume are indicated as the average of readers at a tube current–time level of 300 mAs. The range of largest axial diameters of the 125 nodules was 1.1–8.0 mm (mean, 3.11 ± 1.66 [SD] mm; median, 2.65 mm). The range of nodule volumes was 3.9–397.5 mm³ (mean, 58.1 ± 83.5 mm³; median, 24.9 mm³). Attenuation fluctuated from –800 to 212 HU (mean, –300.4 ± 317 HU; median, –352 HU). Forty-nine of the 125 nodules (39%) were small (< 3 mm), and 66 (53%) were ground-glass nodules (< –300 HU).

The tube current–times levels of 10, 20, 30, 40, and 100 mAs did not exhibit a significant difference in individual reader sensitivity compared with the standard 300 mAs (Table 1). At the lowest tube current level of 5 mAs, only reader 1 had a significant decrease in sensitivity from 82% to 77% (p = 0.0035). Nodule detection by readers 2 and 3 decreased from 98% and 94% at 300 mAs to 95% and 92% at 5 mAs, but the difference was not statistically significant (p = 0.39 and p = 0.45, McNemar test). Individual specificities were high and unaffected by tube current–time level. Specificity at the standard dose was 96%, 89%, and 92% for readers 1, 2, and 3 and did not decrease at lower dose levels (p > 0.31).

Sensitivity as a function of nodule characteristic is shown in Table 2. There were large individual differences in ability to detect individual lung nodules. Reader 1 had significantly lower sensitivity for smaller and ground-glass nodules. The difference in sensitivity for ground-glass versus solid nodules even increased, from 22% at standard dose to 39% at the lowest dose level. The sensitivity of reader 1 for smaller nodules was 12–20% lower than that reader's sensitivity for larger nodules. Interestingly, the sensitivity of reader 2 was not statistically affected by nodule size, and nodule density influenced sensitivity only at dose levels lower than 30 mAs. Reader 3 was influenced by nodule size but not by nodule density. The sensitivity for smaller nodules was 8–14% lower than the sensitivity for larger nodules.

The readers had one thing in common: Sensitivity for all 125 nodules was not significantly influenced by dose level. The tube current–time products exhibited different individual sensitivities (Table 1), which did not decrease significantly from 300 to 5 mAs, according to results of the chi-square test (p > 0.25).

The probability of a true-positive finding if a condition is present (size ≥ 3 mm) compared with the probability of a true-positive finding if the condition is not present (size < 3 mm) resulted in individual reader odds ratios of 3.4–4.2. Only the odds ratio of reader 1 was significant (p = 0.0097). For variable density, the odds ratios were 2.8–9.9, only reader 1 having a statistically significant result (p = 0.00051). The odds ratios for variable tube current were a nonsignificant 1.4–2.1 (p > 0.25). Therefore, the strongest negative effect on sensitivity can be assumed for attenuation less than –300 HU, followed by size less than 3 mm. Examples of small ground-glass nodules at different dose levels are shown in Figure 2.

The most significant predictor of sensitivity was nodule density (p < 0.0001 for all three readers). Nodule size significantly affected the sensitivity of reader 1 only (p = 0.02). Tube current–time level did significantly influence the sensitivity of reader 1 alone (p = 0.007), but for readers 2 and 3, it was not a significant predictor of sensitivity (p = 0.20, p = 0.19) in the logistic regression model.

Agreement of nodule size for lower tube current–time levels compared with reference standard—Agreement between the 5 and 300 mAs levels is presented; the relative volume differences are shown in Figure 3. The results for the other tube current–time levels are listed in Table 3. The mean volume measurement error of two readers combined was –2.2% with a 95% CI of –7.8% to 3.4%; hence a systematic bias between the measurements cannot be assumed (zero is inside the 95% CI). The negative sign means that volume 2 (5 mAs) was measured larger than volume 1 (300 mAs).

The lower limit of agreement was –39% (95% CI, –47% to 31%), and the upper limit was 35% (95% CI, 27–43%). An increase in nodule volume of more than 43% from the standard tube current–time level (300 mAs) to a hypothetical low-dose follow-up examination (5 mAs) is more likely to indicate real nodule growth than interscan variability. Often the nodules were distorted at lower tube current–time levels because of superimposed noise, and their diameter might have increased (Fig. 4) or decreased but would have had no statistical effect on the overall volume measurement. However, the longest axial diameter measurement exhibited significant differences for lower tube current–time levels. The diameters were measured longer for levels of 100 mAs and less (Fig. 4). The mean measurement error for 5 mAs was –5.3% (95% CI, –7.5% to –3%) with corrected limits of agreement of –28% and 17%.

Interreader and intrareader agreement of volume and diameter measurement—With application of the same method for the two readers (reader 1 volume – reader 2 volume / mean volume), the mean interobserver variability of volume measurements at the standard tube current–time level of 300 mAs was 38% (95% CI, 27–49%), meaning a significant difference between the two readers. The mean volume measurement of reader 1 was substantially larger. A lower limit of agreement of –51% (95% CI, –71% to –32%) and an upper limit of agreement of 127% (95% CI, 108–146%) were found. In absolute figures, these results meant a variability among the readers of 9.3 ± 20.7 mm³ with limits of –32.1 mm³ (95% CI, –41 to –23 mm³) to 51 mm³ (95% CI, 42–59.7 mm³) for an average absolute nodule volume of 58.1 mm³ (mean of readers 1 and 2). Interobserver variability did not change substantially for lower tube current–time levels (Table 3). Intraobserver agreement of volume measurement by reader 1 at the standard tube current–time level was 3.9% (95% CI, –2.4% to 10%) with a lower limit of agreement of –35% (95% CI, –43% to –27%) and an upper limit of agreement of 43% (95% CI, 35–51%). The intraobserver measurement error for the longest diameter was 2.8% (95% CI, –0.1% to 5.8% and therefore not significant) and had a smaller limit of agreement according to the single dimension (compared with the volume) of –20% to 26%. The measurement errors for neither volume nor diameter differed substantially for lower tube current–time levels (Table 3).

Agreement on nodule size for lower tube current–time levels compared with standard level—Less agreement of volume and diameter measurement for the computer-aided lesion measurement solution resulted in higher variability and therefore wider limits of agreement. For example, at 5 mAs, the mean volume measurement error was 6% (95% CI, –1.8% to 13.8%) with a lower limit of agreement of –58% (95% CI, –71% to –44%) and an upper limit of agreement of 70% (95% CI, 56–83%).

Interreader and intrareader agreement on volume and diameter measurement—Much better interreader agreement was found for the computer-aided lesion measurement solution than for manual measurements (Fig. 5). The volume measurement error at standard dose was –4.1%, and the corrected limits of agreement were –69% and 60%. In contrast to manual measurement, the lesion measurement solution exhibited no significant difference between the two readers. Good intrareader agreement with a similarly low rate of volume measurement error of –5.9% was found with corrected limits of agreement of –68% and 56%. The reader measurement error for longest diameter also exhibited better interreader and intrareader agreement with the computer-aided technique than without it.

At each dose level, the readers measured a significantly larger volume than did the computer. At the standard dose level (300 mAs) the readers measured 25% more, with corrected limits of agreement of –86% and 136%. The longest diameter was also 15.8% longer measured manually.