|
|
||||||||
Original Research |
1 Department of Biostatistics, Collaborative Health Studies Coordinating Center,
University of Washington, 6200 NE 74th St, Building 29, Suite 310, Seattle, WA
98115.
2 Department of Radiology, Wake Forest University School of Medicine,
Winston-Salem, NC.
3 Los Angeles Biomedical Research Institute at Harbor–UCLA, Torrance,
CA.
Received June 14, 2007;
accepted after revision September 21, 2007.
Supported by contracts N01-HC-95159 through N01-HC-95165 and N01-HC-95169
from the National Heart, Lung, and Blood Institute.
Abstract
|
|
|---|
SUBJECTS AND METHODS. The Multi-Ethnic Study of Atherosclerosis is a prospective cohort study with 6,814 participants 45–84 years old and free of clinical cardiovascular disease at enrollment. Agreement for presence of CAC was assessed for 6,742 participants who had baseline replicate scans on which a CAC score of 0 indicated no coronary calcification. Among 3,380 participants with baseline CAC, the 95% repeatability limits were established with a quantile regression model. Detectable change in CAC during follow-up was defined by an increase or decrease beyond the baseline repeatability limit.
RESULTS. At baseline, 274 (4.1%) of the rescan pairs were discordant
(presence or absence of CAC). Greater body mass index was associated with a
discordant pair (trend, p < 0.05). The upper 95% repeatability
limits were (0.17 x Agatston score) + (4.89 x
Agatston
score) + (0.44 x body mass index)–10.84 for Agatston score and
(0.16 x volumetric calcium score) + (4.30 x
volumetric
calcium score) + (0.23 x body mass index)–5.00 for volumetric
calcium score. Rescan repeatability was comparable for electron beam and
4-MDCT scanners. At 2.5 years of average follow-up (range, 0.9–5.0
years), a detectable increase in Agatston and volumetric calcium scores was
observed in 1,027 (36.3%) and 1,020 (36.0%), respectively, of 2,832
participants with baseline CAC.
CONCLUSION. The repeatability limits derived can be used to evaluate whether an increase in CAC score exceeds that expected from measurement error alone.
Keywords: cardiac CT coronary artery calcification detectable change repeated scan repeatability repeatability limit
|
|
|---|
The Multi-Ethnic Study of Atherosclerosis [10] is a prospective cohort study with 6,814 U.S. participants free of clinical cardiovascular disease. In the study, two sequential CT scans were acquired at enrollment with EBCT or 4-MDCT for nearly all participants (between years 2000 and 2002) [11]. The first follow-up CT examination was performed 1–5 years after the first. The Multi-Ethnic Study of Atherosclerosis affords an excellent opportunity to compare rescan variability across CT type and participant characteristics (body size, sex, race) and to explore temporal change in CAC exceeding that expected on the basis of baseline rescan variability. The objectives of our study were to construct 95% repeatability limits of CAC scores and to use the repeatability limits to examine detectable change in CAC over time in the Multi-Ethnic Study of Atherosclerosis.
|
|
|---|
Multi-Ethnic Study of Atherosclerosis CT Examinations
A standardized Multi-Ethnic Study of Atherosclerosis CT and reading
protocol has been documented
[11,
12]. Briefly, participants
were instructed about breath-holding and about the importance of immobility
during scanning. All scans were acquired in a single breath-hold. Participants
were asked to be relaxed on the table between scans. The protocol was designed
to require less than 15 minutes in the CT room, and the average time between
scans at baseline was 2 minutes. At the Multi-Ethnic Study of Atherosclerosis
baseline, EBCT (Imatron C150 scanner, GE Healthcare) was used at three study
sites, and 4-MDCT (LightSpeed scanner, GE Healthcare, or Volume Zoom scanner,
Siemens Medical Solutions) at three sites. The field of view of CT images was
350 mm. The detailed CT acquisition has been described by Carr et al.
[11]. A total of 6,742
participants had a pair of CT scans with valid Agatston and volumetric calcium
scores at baseline, including 3,380 participants with CAC (i.e., a positive
CAC score).
Follow-up CAC measurements were performed on one half of the cohort (randomly selected) at a second Multi-Ethnic Study of Atherosclerosis examination in years 2002–2004 and the other half of the cohort at a third examination in years 2004–2005, an average of 1.6 and 3.2 years, respectively, after the baseline examination. A total of 5,757 participants in the Multi-Ethnic Study of Atherosclerosis had a follow-up CAC score measured at either examination 2 or examination 3, and 5,733 of them underwent scanning twice consecutively. During the follow-up period, scanners were replaced with newer models at some study sites. At examinations 2 and 3, the following scanners were used for some of the study participants: LightSpeed Pro 16 (GE Healthcare) (n = 515), Sensation 16 (Siemens Medical Solutions) (n = 269), Sensation 64 (Siemens Medical Solutions) (n = 186), Aquilion 32 (Toshiba) (n = 39), and Aquilion 64 (Toshiba) (n = 130).
In our study, we focused on the Agatston score [13] and the volumetric calcium score [14]. Calibration phantoms with known calcium density were used in the Multi-Ethnic Study of Atherosclerosis to adjust for attenuation variability across study sites and participants [12]. In our study, however, we used raw CAC measurements without phantom calibration because the within-subject variability was of primary interest and because the Multi-Ethnic Study of Atherosclerosis phantom calibration algorithm is not used clinically. Our findings with raw CAC measurements would be easier to generalize to scans obtained in clinical practice.
Detectable Change in CAC over Time
For participants with CAC at baseline, we defined detectable change in CAC
as a change exceeding the baseline repeatability limits. That is, a difference
between the two CAC measurements exceeding that expected from measurement
error alone. We examined detectable CAC change from baseline to follow-up
using data on participants who had CAC at baseline and who had CAC follow-up
data. At each time point (baseline or follow-up), a point estimate of the
amount of calcium in each participant was presented as an average CAC score,
mostly from dual-scan runs.
Statistical Analysis
Given two repeated scans (scan 1, scan 2), we used M to denote the
average magnitude, or (scan 1 + scan 2)/2; D, the difference, (scan
1–scan 2); and |D|, the absolute value of the
difference. Relative absolute difference is defined as the absolute difference
divided by the average (A) with the result expressed as a percentage,
or (|D|/A) x 100.
The distribution of CAC in asymptomatic participants was highly skewed, with a considerable proportion of participants having no measurable calcified plaque according to the scoring criteria (CAC score, 0). Among the Multi-Ethnic Study of Atherosclerosis participants, approximately 50% of participants had no CAC at baseline. Therefore, as in previous reports of CT in the Multi-Ethnic Study of Atherosclerosis [11, 15], we reported the proportions of discordant rescan pairs (presence or absence of CAC) using logistic regression models to examine whether this disagreement was associated with CT scanner type or participant characteristics. Body mass index (BMI) (weight in kilograms divided by height squared in meters) was used as a measure of body size. When appropriate, we further examined other indicators of body size, including weight, waist circumference, waist-to-hip ratio, and body surface area.
In participants with CAC at baseline, we first generated a Bland-Altman
plot [8] for each calcium
measurement and found a nonuniform variability of CAC measurements. We
estimated the 95th percentile for the absolute difference
(|D|) as a function of the average magnitude
(M) and other covariates (e.g., BMI) by fitting a quantile regression
model [16]. Results of
previous work [9,
17] suggested that the rescan
repeatability of CAC measurements is a function of square-root-transformed
magnitude (
M), so we included this value as a term in our
quantile regression model. A common way
[9,
18] of estimating CAC
repeatability limits is to use the half-normal method, in which it is assumed
that properly scaled differences follow a normal distribution. The quantile
regression method we used has the advantage of not requiring a distributional
assumption.
Comparisons of proportions and tests for interaction were conducted with
logistic regression (Wald test). Tests of the linear trend across BMI
categories were performed by treating the variable as continuous. A value of
p
0.05 was considered significant. All analyses were performed
with Stata version 9.2 software (Stata).
|
|
|---|
|
|
|
To estimate the repeatability limits for Agatston score, we initially
modeled the repeatability limit of rescan absolute difference
(|D|) using a square-root-transformed rescan average
(
M) based on the nonlinear relation between
|D| and M and on previous results
[9,
17]. There remained, however,
a significant residual dependence between the resulting repeatability limit
and CAC magnitude (p = 0.01). Thus we included in the quantile
regression model a second order of
M, which is identical to
M; that is, (
M)2 = M. The
resulting repeatability limit (Agatston score) was estimated as:
![]() |
A total of 168 (5.0%) of the observations were outside this repeatability
limit, and this proportion was not associated with CAC magnitude (p =
0.88). We examined the associations of scanner type and participant
characteristics with this repeatability limit
(Table 2) and found a
significant association with BMI. We therefore added an adjustment term for
BMI in the quantile regression model. The resulting repeatability limit
(Agatston score) was expressed as follows:
![]() |
|
With this model, the proportion of observations falling outside the repeatability limits was not associated with CAC magnitude (p = 0.89), BMI, or the other factors listed in Table 2.
Figure 3 compares the CAC-adjusted and CAC- and BMI-adjusted repeatability limits for Agatston score. The plot shows that these two repeatability limits agreed well with each other. Given the ranges of Agatston scores (from zero to several thousand) and BMI (generally between 15 and 60), the effect of BMI on repeatability is minimal. However, for persons with very small amounts of CAC (e.g., < 30 Agatston units), the effect of BMI correction is more noticeable.
|
![]() |
![]() |
The proportions outside these repeatability limits were not associated with CAC magnitude (p = 0.77 and p = 0.73, respectively). Table 2 shows the effects of scanner type and participant characteristics on these repeatability limits. The rescan differences in volumetric calcium score appeared to be less affected by BMI than they were in the Agatston score.
Table 3 shows the CAC-adjusted repeatability limits, or detectable change, for Agatston and volumetric calcium scores. For example, for a participant with a baseline score of 50 Agatston units, a change of 45 units or more is unlikely to be due to measurement error. In contrast, for a baseline Agatston score of 300, changes of more than 139 units are unlikely to be due to measurement error alone.
|
|
Applying the CAC-adjusted repeatability limits to the observed changes in 2,832 participants with positive baseline CAC scores, we found that 1,031 (36.4%) and 1,027 (36.3%) of the participants had a detectable CAC increase for Agatston and volumetric calcium scores, respectively. As expected, the proportion with detectable change increased with time since baseline (Fig. 4). When the CAC- and BMI-adjusted repeatability limits were applied, 1,007 and 1,020 participants had a detectable CAC increase for Agatston and volumetric calcium scores, respectively. A total of 913 participants had a detectable CAC increase with respect to all four repeatability limits considered.
Only 11 of the 313 participants with a negative change in CAC score had a detectable decrease in CAC with respect to at least one repeatability limit. Four of the 11 participants had a moderate amount of CAC at baseline (Agatston score > 40; volumetric calcium score > 30) but had no CAC at follow-up. In particular, one participant had coronary revascularization between baseline and follow-up CT scans (baseline Agatston score, 294; follow-up Agatston score, 138).
|
|
|---|
We found that greater BMI is associated with a measurable decline in rescan repeatability (disagreement for presence or absence, wide repeatability limits). However, this body size effect would be practically meaningful only for persons with low CAC scores. In other words, early detection of true calcification with CT and current CAC scoring methods is more difficult for extremely overweight people (e.g., BMI > 40). Similar findings were obtained in CT phantom data analysis [12, 19, 20]. Possible explanations include soft-tissue attenuation [19], background image noise [21], and body tissue composition.
The baseline repeatability limits were used to quantify detectable change in CAC, and this approach has several advantages. The proportion of scans with detectable change increases with length of time between scans, as expected. In addition, the number of participants with detectable regression is very small (11 of 313 participants with a negative change), and in many of those cases the regression appears not to be due to measurement error.
Although we present results for raw CAC scores, the rescan variability of the phantom-adjusted CAC scores was almost identical to that of raw CAC scores. Calibration with the phantom did not entirely eliminate rescan variability due to obesity; however, this BMI effect was minimal compared with the effect of CAC magnitude.
We found no difference in rescan variability for the scanner types (EBCT and 4-MDCT) used in the Multi-Ethnic Study of Atherosclerosis baseline CT examinations between 2000 and 2002. We also found that 3–5% of the rescan difference with new MDCT scanners used at follow-up (examinations 2 and 3) was beyond our 95% repeatability limits, but the relatively small number of participants imaged with the new scanners must be considered. The CT data were obtained according to a standardized protocol for the Multi-Ethnic Study of Atherosclerosis, and we put our efforts into controlling for participant-specific covariates (most important, CAC magnitude). Nevertheless, ability to generalize the Multi-Ethnic Study of Atherosclerosis baseline 95% repeatability limits to different scanners from different vendors is limited, and further study is warranted.
A CAC score captures only part of the available pathobiologic information on calcified plaque available from cardiac CT images. Global CAC scores of the entire coronary circulation can mask or minimize substantial specific changes in vessels and plaque. In addition, different positions on the scanner table at different times can influence CT images and subsequent CAC scores. Thus an increase in CAC score over time does not necessarily imply clinical atherosclerotic progression and subsequent increased risk of cardiovascular events. The prognostic utility of tracking change in coronary calcified plaque remains unclear with limited evidence [6]. In evaluation of the prognostic utility of tracking calcification, understanding of and possible adjustment for within-person variability will likely be useful. The use of baseline rescan repeatability information may be one way to increase the sensitivity and specificity of change in calcification.
We emphasize the importance of controlling for CAC magnitude in repeatability analyses. The distribution of CAC differs substantially with age, sex, and race and ethnicity [22]. Unless the ranges of CAC magnitude are comparable, simple comparison of either absolute difference or relative difference will not be useful in comparisons of rescan variability.
Acknowledgments
We thank the investigators, staff, and participants of the Multi-Ethnic
Study of Atherosclerosis. A full list of participating Multi-Ethnic Study of
Atherosclerosis investigators and institutions is available at
www.mesa-nhlbi.org.
|
|
|---|
This article has been cited by other articles:
![]() |
R. J. Gibbons, P. A. Araoz, and E. E. Williamson The year in cardiac imaging. J. Am. Coll. Cardiol., January 6, 2009; 53(1): 54 - 70. [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |