| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Original Research |
1 Departments of Radiology and Epidemiology & Biostatistics, Erasmus Medical
Center, Dr. Molewaterplein 50, Rm. Ee 2118, 3015 GE Rotterdam, The
Netherlands.
2 Department of Radiology, Maastricht University Hospital, Maastricht, The
Netherlands.
Received August 18, 2004;
accepted after revision November 23, 2004.
This study was supported by a grant (nr. 945-01-039) from Zon Mw,
Netherlands Organization for Health Research and Development, and by a grant
(904-66-091) from The Netherlands Organization for Scientific Research, The
Hague, The Netherlands.
Abstract
 Top Abstract IntroductionSubjects and MethodsResultsDiscussionReferences |
|---|
SUBJECTS AND METHODS. Of 226 eligible patients, 69 were excluded.
The remaining 157 consecutive patients were prospectively randomized to either
MR angiography (n = 78) or MDCT angiography (n = 79). Two
observers independently evaluated for arterial stenosis or occlusion on MR
angiography (2,157 segments) and MDCT angiography (2,419 segments) using a
5-point ordinal scale. Vessel wall calcifications were noted. Interobserver
agreement for each technique was evaluated with a weighted kappa
(
w) statistic.
RESULTS. Although interobserver agreement for both was excellent,
the interobserver agreement for MR angiography (w = 0.90;
95% confidence interval [CI], 0.89–0.92) was higher than that for MDCT
angiography (w = 0.85; 95% CI, 0.83–0.86) for
reporting the degree of arterial stenosis or occlusion in all segments. For
the different anatomic locations, the interobserver agreement for MR
angiography versus MDCT angiography was as follows: aortoiliac
(w =0.91 vs 0.84, respectively), femoropopliteal
(w = 0.91 vs 0.87), and crural (w = 0.90
vs 0.83) segments. The interobserver agreement of MDCT angiography
significantly decreased in the presence of calcifications but was still good
for all anatomic locations. The lowest agreement was found for crural segments
in the presence of calcifications (w = 0.67). With MR
angiography, there were 12 times more nondiagnostic segments than with MDCT
angiography (81 vs 7, respectively).
CONCLUSION. Interpretations of MR angiography and MDCT angiography for peripheral arterial disease have an excellent interobserver agreement. MR angiography has a higher interobserver agreement than MDCT angiography, and the presence of calcified segments significantly decreases interobserver agreement for MDCT angiography.
|
TopAbstractIntroductionSubjects and MethodsResultsDiscussionReferences |
|---|
Both contrast-enhanced 3D MR angiography and MDCT angiography are increasingly used for noninvasive vascular imaging. MR angiography has gained widespread use for imaging peripheral arterial disease [5–7]. Disadvantages of MR angiography include difficulty in depicting small vessels because of the limited spatial resolution and a tendency to overestimate the degree of stenosis because of signal intensity loss in tightly stenotic lesions [8].
The recently introduced MDCT scanners have substantially improved MDCT angiography for peripheral arterial disease. The use of MDCT technology has resulted in shorter acquisition time, increased volume coverage, lower dose of contrast medium, and improved spatial resolution [9, 10]. Results of several studies have shown that MDCT angiography is accurate for imaging peripheral arteries [11–16]. The main disadvantages of MDCT angiography are the use of radiation, the use of potentially nephrotoxic iodinated contrast medium, the time-consuming 3D reconstruction techniques, and the difficulty in assessing arterial lumen stenosis in the presence of vessel wall calcifications [17–19].
In the evaluation of new diagnostic tests, the study of its interobserver agreements plays an important role [20]. The accuracy of a test can never be perfect if assessments by different observers show significant variation. Furthermore, it is likely that poor interobserver agreement can cause variation in clinical decision making. Thus, apart from evaluating accuracy in comparison with a reference standard, it is important to evaluate reproducibility, including interobserver agreement for test results.
The purpose of this study was to compare the interobserver agreement for interpretations of contrast-enhanced 3D MR angiography and MDCT angiography in patients with peripheral arterial disease.
|
TopAbstractIntroductionSubjects and MethodsResultsDiscussionReferences |
|---|
Inclusion criteria for participation in the study were patient age of older than 18 years, symptomatic peripheral arterial disease, an ankle-brachial index of less than 0.90, and referral for diagnostic imaging workup to evaluate the feasibility of a revascularization procedure. Exclusion criteria included contraindications for MR angiography (e.g., pacemaker or claustrophobia) or MDCT angiography (e.g., severe renal insufficiency or adverse reactions to iodinated contrast agent) and the necessity of an acute intervention. The subjects were randomized across two diagnostic strategies consisting of MR angiography and MDCT angiography. The study was approved by the institutional review board, and informed consent for the study and all articles derived from the study was obtained from all patients.
MR Angiography
All examinations were performed with a 1.5-T imager (Signa, GE Healthcare)
that was equipped with echo-speed gradients (40 mT/m, 150 mT/m/msec). A
dedicated peripheral vascular phased-array coil was used for signal
reception.
For bolus-chase MR angiography, commercially available software (SmartStep, GE Healthcare) was used. The imaging protocol included the following imaging procedures. First, localizer MR images were obtained with transverse time-of-flight (TOF) scout views of three locations: aortoiliac, femoropopliteal, and crural segments. The parameters for the TOF sequence were as follows: TR/TE, 23/4.4; flip angle, 70°; bandwidth, 15.63 kHz; slice thickness, 8.0 mm; and matrix, 256 x 128. Acquisition times were between 1.31 and 2.37 min.
On the basis of the localization study, three MR angiographic volumes were prescribed that covered the abdominal and lower extremity vasculature. Then a contiguous 3D MR angiographic mask image was acquired with integrated automated table movement. The imaging parameters for acquisition of the mask images were identical to those used for acquisition of the contrast-enhanced images. The following parameters were used for imaging the aortoiliac location: TR/TE, 4.8/1.5; flip angle, 30°; field of view, 400 x 280 mm; slice thickness, 2.6 mm; matrix, 384 x 192; and phase-encoding, centric. The parameters for imaging the femoropopliteal location were as follows: TR/TE, 4.8/1.5; flip angle, 30°; field of view, 400 x 320; slice thickness, 2.0 mm; matrix, 384 x 192; and phase-encoding, centric. The parameters for imaging the crural location were as follows: TR/TE, 5.1/1.5; flip angle, 30°; field of view, 400 x 360 mm; slice thickness, 2.0 mm; matrix, 512 x 512; and phase-encoding, elliptic centric. Zero interpolation was performed with adding of extra zeros to the k-space in all three planes before Fourier transformation to improve image quality.
After the mask acquisitions, contrast-enhanced imaging was started at the aortoiliac location, which was automatically initialized after automated bolus detection (SmartPrep, GE Healthcare). Subsequently, 3D MR angiographic images of the femoropopliteal and crural locations were acquired sequentially.
The contrast agent (gadopentetate dimeglumine [Magnevist, Schering]) was administered in the antecubital vein using a 20-gauge IV catheter. Each patient received 45 mL of contrast agent (0.5 mmol/mL) at a rate of 1.2 mL/sec for the first 10 mL and 0.8 mL/sec for the remaining 35 mL (total injection duration, 52 sec), followed by a saline flush of 15 mL at a rate of 0.8 mL/sec. For this dual-phase injection, an automated injector (Spectris, Medrad) was used to ensure precise contrast agent injections.
We used a subtraction technique before maximum-intensity-projection (MIP) reconstructions were performed. The mask images were subtracted from the contrast-enhanced images at each station. Twelve rotated volume MIP images ranging from –90° to 90° were reconstructed for each subtracted data set. These volume MIP images were documented on film and sent together with the source data to a remote workstation (Advanced Windows 3.1, GE Healthcare).
MDCT Angiography
MDCT angiography was performed on a Sensation 16 scanner (Siemens Medical
Solutions). Patients were in the supine position on the CT table with their
legs held together.
After an initial scout image (120 kV, 100 mAs) was obtained, the scanning range was planned to encompass the entire vascular system from the diaphragm to the level of the ankles. For optimal intraluminal contrast enhancement, the delay time between the start of contrast material administration and the start of scanning was obtained for each patient individually using a bolus-tracking technique (CARE-Bolus, Siemens Medical Solutions). Subsequently, a nonionic contrast material (iodixanol [320 mg I/mL Visipaque, Amersham Health]) was administered through a 20-gauge cannula that was placed in the patient's antecubital vein for a total volume of 120 mL.
The contrast material was administered with an automatic power injector (EnVision CT, Medrad) at a flow rate of 3 mL/sec. Ten seconds after the start of contrast material administration, a series of dynamic low-dose monitoring scans (120 kV, 20 mAs, 0.5-sec scanning time, 1.25-sec interscan delay) were obtained. After the preset attenuation of 100 H above the baseline attenuation was reached, the CT scan was automatically triggered. Data acquisition was performed craniocaudally with the following parameters: collimation, 0.75 mm; number of detector rows, 16; table feed, 18 mm per rotation; gantry rotation period, 0.5 sec; pitch, 1.5; X-ray tube voltage setting, 120 kV; and current, 140 mAs.
Transverse sections were reconstructed with a 2-mm slice thickness at an interval of 1 mm. Two orthogonal curved planar reformations were created along the longitudinal axis of the aorta through both common and external iliac arteries and the common femoral artery using commercially available software on the CT console. All data were then transferred to a dedicated workstation (Easy Vision, Philips Medical Systems) that allowed postprocessing of the images.
The reconstructions were performed by one of two technologists experienced in 3D postprocessing and segmentation techniques. Segmentation was performed of both bone structures and vessel wall calcifications resulting in images containing the contrast-enhanced vascular lumen without vessel wall calcifications and bones. Of these data sets, rotating volume MIP images were generated using the commercially available software installed on the workstation. This resulted in 12 angiogramlike images rotating more than 180° for aortoiliac, femoropopliteal, and crural arteries.
Image Analysis
All MR and MDCT angiograms were interpreted independently by two observers.
Observer 1 is a vascular radiologist, and observer 2 is a dedicated researcher
with 2.5 years of general radiology training and 1 year of experience in
vascular radiology. Both observers have extensive experience in interpreting
MR angiography and MDCT angiography.
The volume MIPs of both MR angiography and MDCT angiography and the curved planar reconstructions of MDCT angiography were printed. The reconstructed coronal MR and transverse CT images (source data), along with the standardized volume MIPs and curved planar reconstructions, were available for both observers on dedicated workstations. For image interpretation, the observers used both the hard copies and the source data on the workstations. In almost all cases, the observers used the source data. In a few cases—that is, if the volume MIPs provided a clear-cut map of all vessels—the source data were not used.
For analysis purposes, the arterial vascular system was divided into three anatomic locations to include a total 31 segments—namely, the aortoiliac arteries consisting of the distal aorta, paired common iliac arteries, and external iliac arteries; the femoropopliteal arteries consisting of the paired common femoral arteries, deep femoral arteries, superficial femoral arteries (proximal and distal parts), and popliteal arteries (above and below the knee); the crural arteries consisting of the paired anterior tibial arteries (proximal and distal parts), tibial peroneal trunk, posterior tibial arteries (proximal and distal parts), and peroneal arteries (proximal and distal parts).
Segments not contained within the imaging volume or not interpretable because of venous enhancement or artifacts were considered nondiagnostic. All other segments were assessed for the presence of stenotic disease. The following 5-point ordinal scale was used to grade stenotic or occlusive disease: zero for 0–19% stenosis; 1, 20–49% stenosis; 2, 50–74% stenosis; 3, 75–99% stenosis; and 4, occlusion. When two or more stenotic luminal lesions were detected in the same vessel segment, the most severe lesion was used for grading and analysis. The observers recorded the presence of vessel wall calcifications on MDCT angiography and recorded nondiagnostic segments on both MR angiography and MDCT angiography.
Statistical Analysis
Interobserver agreement was determined by calculating a weighted kappa
(w) statistic, which takes the degree of disagreement into
account and accounts for differences in the importance of disagreement. The
kappa statistic indicates the agreement beyond chance. Strength of agreement
can be interpreted as poor ( < 0.20), fair ( =
0.21–0.40), moderate ( = 0.41–0.60), good ( =
0.61–0.80), or excellent ( = 0.81–1.0)
[21]. If a segment was
classified as nondiagnostic by at least one of the observers, the segment was
omitted from the weighted kappa calculations for reporting the degree of
arterial stenosis. In addition, the percentage of overall agreement including
the nondiagnostic segments was calculated. We used an unweighted kappa
statistic to calculate the interobserver agreement for classifying
nondiagnostic versus diagnostic segments.
The kappa values can be expected to be higher if legs without symptoms are included in the analysis because it is likely that in nondiseased segments interobserver agreement is higher. Therefore, we performed a second analysis in which we included only the most symptomatic leg of each patient. If symptoms were the same in both legs, we randomly selected one leg. In each symptomatic leg, one segment per anatomic location was randomly selected. Thus, in this secondary analysis, the number of segments analyzed was reduced from 31 to three segments per patient (i.e., one aortoiliac segment, one femoropopliteal segment, and one crural segment).
Additional analyses were performed to examine the effect of vessel wall calcifications, the effect of disease severity (claudication vs critical ischemia), and the effect of learning during the trial period (first half vs second half of the trial period). An unweighted kappa statistic was used to calculate the interobserver agreement for diagnosing hemodynamically insignificant (i.e., stenosis < 50%) versus significant (i.e., stenosis 50%) arterial stenoses and for determination of nonoccluded (i.e., stenosis < 99%) versus occluded arteries. Calculations were performed with statistical packages (SPSS [version 11.0], Statistical Package for the Social Sciences; and SAS [version 8.2], SAS Institute) for Windows (Microsoft).
|
|
TopAbstractIntroductionSubjects and MethodsResultsDiscussionReferences |
|---|
Of the 78 patients assigned to MR angiography, 73 actually underwent MR angiography. Three patients underwent digital subtraction angiography because of unknown claustrophobia (n = 2) and the necessity of an acute intervention due to progressive disease (n =1). One patient underwent MDCT angiography because of logistical problems. One patient refused MR angiography and underwent no other diagnostic test. Of the 79 patients allocated to undergo CT angiography, all underwent MDCT angiography. The baseline characteristics of the patients are described in Table 1.
|
In the group of patients who underwent MR angiography, 2,238 segments were imaged. Twenty-five segments were not imaged because of lower leg amputation (n = 23) and congenital agenesis of the tibioperoneal trunk (n = 2). On MR angiography, 81 segments were nondiagnostic, which were mainly crural segments and mainly due to venous enhancement. This leaves 2,157 segments in the MR angiography group on which the analyses for reporting the degree of arterial stenosis is based. In the MDCT angiography group, 2,426 segments were imaged. Twenty-three segments were not imaged because of lower leg amputation (n = 17) and congenital agenesis of the tibioperoneal trunk (n = 6). On MDCT angiography, only seven segments were nondiagnostic, which were due to a total knee arthroplasty, leaving 2,419 segments on which the analyses for reporting the degree of arterial stenosis is based.
The interobserver agreement for reporting the degree of arterial stenosis
or occlusion in all segments was statistically significant lower for MDCT
angiography (w = 0.85; 95% confidence interval [CI],
0.83–0.86) than for MR angiography (w = 0.90; 95% CI,
0.89–0.92, p < 0.001). Nevertheless, there was excellent
interobserver agreement of both MR angiography and MDCT angiography for
reporting the degree of arterial stenosis or occlusion in all segments (Tables
2 and
3). The percentage of overall
agreement was 89% (95% CI, 88–90%) for MR angiography and 83% (95% CI,
81–85%) for MDCT angiography (Tables
2 and
3). The results of the primary
and secondary analyses were similar in the different anatomic locations
(Tables 4 and
5).
|
|
|
|
Interobserver agreement for classifying nondiagnostic versus diagnostic
segments was excellent for MDCT angiography ( = 0.92; 95% CI,
0.77–1.00) and was good for MR angiography ( = 0.70; 95% CI,
0.60–0.79). The interobserver agreement of MDCT angiography for
reporting the degree of arterial stenosis or occlusion in all segments was
statistically significantly lower for arterial segments with vessel wall
calcifications than for segments without vessel wall calcifications
(Table 6). In the presence of
vessel wall calcifications there was still good interobserver agreement in all
anatomic locations. The lowest agreement was found for crural segments in the
presence of calcifications (w = 0.67; 95% CI,
0.60–0.73).
|
Subgroup analyses for disease severity (claudication vs critical ischemia)
showed no difference in interobserver agreement of MR angiography
(w = 0.90 vs 0.92, respectively) and MDCT angiography
(w = 0.84 vs 0.86, respectively)
(Table 7). Subgroup analyses
for trial period (first half vs second half) showed no difference in
interobserver agreement of MR angiography (w = 0.91 vs 0.89,
respectively). Interobserver agreement of MDCT angiography was lower for the
second half of the trial period (w = 0.88 vs 0.81,
respectively) (Table 7).
|
Interobserver agreement for diagnosing hemodynamically insignificant
stenosis versus significant arterial stenosis was excellent for both MR
angiography ( = 0.92; 95% CI, 0.91–0.94) and MDCT angiography
( = 0.86; 95% CI, 0.84–0.89). Interobserver agreement for
determination of nonoccluded versus occluded arteries was excellent for both
MR angiography ( = 0.94; 95% CI, 0.92–0.96) and MDCT angiography
( = 0.91; 95% CI, 0.88–0.93).
|
TopAbstractIntroductionSubjects and MethodsResultsDiscussionReferences |
|---|
Literature about interobserver agreement of MR angiography and MDCT angiography in patients with peripheral arterial disease is scarce. Most articles describe agreement only between two different techniques and if they report interobserver agreement, it is difficult to compare with our results because they often use Cohen's kappa statistic instead of the weighted kappa statistic as we did. For MR angiography, we only found one weighted kappa value of 0.86 representing interobserver agreement in all segments [5]. For MDCT angiography, we did not find any weighted kappa values in the literature. For digital subtraction angiography a weighted kappa value of 0.87 has been reported [22], which is similar to our results for MR angiography and MDCT angiography.
Our expectation that kappa values could have been overestimated by including nondiseased segments was not confirmed by the secondary analysis. In fact, including only the most symptomatic leg of each patient with only one segment per station yielded kappa values very similar to those of the primary analysis (Tables 4 and 5). Furthermore, in the primary and secondary analyses, we found excellent interobserver agreement for both MR angiography and MDCT angiography in all locations.
Vessel wall calcifications on MDCT angiograms have been shown to affect image interpretation in several studies [19, 23, 24]. In our experience, extensive arterial wall calcifications of aortoiliac, femoropopliteal, and crural arteries are frequently seen in patients with peripheral arterial disease and interfere with image interpretation (Figs. 2A, 2B, and 2C). The small vessel diameter combined with vessel wall calcifications may have contributed to the lowest agreement, which occurred in the crural arteries. It is important to note that the apparent obscuration of the arterial lumen by vessel wall calcifications strongly depends on the window settings. Therefore, adjusting the window settings is a way to minimize "blooming" of calcium. Despite the impairment of vessel analysis in the presence of vessel wall calcifications, the possibility of localizing arterial wall calcifications that may have therapeutic relevance may be an advantage of MDCT angiography [25].
|
|
|
We expected that data might be affected by a learning effect during the trial period, which would result in an increase of the interobserver agreement over time. To evaluate whether this effect was present, we divided our data in two groups representing the first half and second half of the trial period. Although this method does not reflect a true learning process on a case-by-case basis, it nevertheless is useful to document changes in data perception with increasing experience. For MR angiography, interobserver agreement was similar for both the first half and second half of the trial period. For MDCT angiography, the interobserver agreement decreased during the trial period. A possible explanation for our findings may be that the observers became less meticulous in distinguishing between grade zero (0–19% stenosis) and 1 (20–49% stenosis) or between grade 2 (50–74% stenosis) and 3 (75–99% stenosis) stenoses due to routine and the realization that these distinctions are less important clinically. This was especially true given that image interpretation is more time-consuming for MDCT angiography than for MR angiography due to the many source images and because image interpretation is hampered by vessel wall calcifications.
We acknowledge several limitations of our study. First, we did not study intraobserver agreement, although this information may have been interesting. However, because the interobserver agreement was excellent, it is likely that the intraobserver agreement is excellent too. A second possible limitation relates to the measurement of the severity of stenoses. The degree of stenoses was not measured with electronic or manual calipers, which as a consequence introduces a more subjective judgment. Quantitative computerized assessment of the degree of stenosis reduces interobserver variability [26]. On the other hand, our scoring system, which was visual assessment, may well reflect daily clinical practice in our hospital and probably other hospitals in which computerized quantitative measurement is not yet used on a routine basis.
An additional limitation is that we had a high number of nondiagnostic segments on MR angiography. Performing an initial high-resolution MR angiography sequence of the tibial vessels may reduce the number of nondiagnostic segments. Furthermore, all nondiagnostic segments were omitted from the calculations of the weighted kappa values for reporting the degree of arterial stenosis. We omitted the nondiagnostic segments because there is no meaningful and logical position in which the nondiagnostic category can be included in the categoric scale for grading arterial stenosis. Furthermore, our method of analysis is consistent with daily clinical practice in which a nondiagnostic segment will be ignored or reimaged before final treatment is planned. We acknowledge that it is important to have information about nondiagnostic segments and therefore we reported how many segments were nondiagnostic on MR angiography and MDCT angiography and calculated the percentage of overall agreement, which included the nondiagnostic segments. Furthermore, we calculated an unweighted kappa value for classifying segments as nondiagnostic versus diagnostic.
Finally, we did not include the dorsal pedal and the plantar arteries in the kappa calculations because the foot is not routinely examined with MR angiography and MDCT angiography in our hospital.
In conclusion, the results of our study show that interpretation of MR angiography and MDCT angiography for peripheral arterial disease has an excellent interobserver agreement, MR angiography has a higher interobserver agreement than MDCT angiography, and calcified segments on MDCT angiography significantly decrease interobserver agreement. These results support the increasing use of both MR angiography and MDCT angiography in the diagnostic imaging workup of patients with peripheral arterial disease.
|
TopAbstractIntroductionSubjects and MethodsResultsDiscussionReferences |
|---|
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
This article has been cited by other articles:
|
|
 |
|
  M. H. Heijenbrok-Kal, M. C. J. M. Kock, and M. G. M. Hunink Lower Extremity Arterial Disease: Multidetector CT Angiography Meta-Analysis Radiology, November 1, 2007; 245(2): 433 - 439. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. Ouwendijk, M. C. J. M. Kock, L. C. van Dijk, M. R. H. M. van Sambeek, T. Stijnen, and M. G. M. Hunink Vessel Wall Calcifications at Multi-Detector Row CT Angiography in Patients with Peripheral Arterial Disease: Effect on Clinical Utility and Clinical Predictors Radiology, November 1, 2006; 241(2): 603 - 608. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
