| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Original Research |
1 Intestinal Imaging Centre, St. Mark's Hospital, Watford Rd., Level 4V, Harrow,
Middlesex, HA1 3UJ, United Kingdom.
2 Paul Strickland Scanner Centre, Mount Vernon Hospital, Northwood, United
Kingdom.
Received May 27, 2004;
accepted after revision September 30, 2004.
Supported by a grant from the Royal College of Radiologists, London, United
Kingdom.
Abstract
 Top Abstract IntroductionSubjects and MethodsResultsDiscussionReferences |
|---|
SUBJECTS AND METHODS. Thirty-one patients (17 men, 14 women; median age, 69 years) with proven colorectal cancer were examined prospectively using MDCT. A 65-sec dynamic study (cine mode, 4 x 5 mm collimation) was acquired through the tumor after IV contrast administration (100 mL of iopamidol 350, 5 mL/sec). Tumor blood volume, blood flow, mean transit time, and permeability measurements were determined by two independent observers using commercial software. Inter- and intraobserver agreement was assessed using the Bland-Altman test.
RESULTS. The mean difference for interobserver agreement (95% limits of agreement) was -0.81 mL/100 g tissue (-3.14 to 1.52); -9.94 mL/100 g tissue/min (-51.43 to 32.65); -1.09 sec (-7.05 to 4.86); and -2.90 mL/100 g tissue/min (-11.48 to 5.68) for blood volume, blood flow, mean transit time, and permeability, respectively. The intraclass correlation coefficient was 0.83, 0.89, 0.89, and 0.80, respectively. The mean difference for intraobserver agreement (95% limits of agreement) was 0.12 mL/100 g tissue (-1.90 to 2.14); 0.02 mL/100 g tissue/min (-13.13 to 13.17); -0.19 sec (-3.19 to 2.81); and 0.00 mL/100 g tissue/min (-2.45 to 2.45) for observer 1 and 0.26 mL/100 g tissue (-1.46 to 1.98); 4.47 mL/100 g tissue/min (-26.65 to 35.59); -0.21 sec (-2.48 to 2.06); 1.08 mL/100 g tissue/min (-4.92 to 7.08) for observer 2. The intraclass correlation coefficient was 0.86, 0.98, 0.97, 0.98 for observer 1 and 0.93, 0.96, 0.99, and 0.94, respectively, for observer 2.
CONCLUSION. There is greater inter- than intraobserver agreement for CT vascular perfusion measurements of primary colorectal cancer, which must be addressed for reliable clinical application in therapeutic monitoring.
|
TopAbstractIntroductionSubjects and MethodsResultsDiscussionReferences |
|---|
Prognosis is related to tumor stage at presentation, with an approximate 5-year survival rate of 80% for patients with class A cancers according to Dukes' classification compared with 35% for Dukes' class C tumors [7]. Local tumor stage predicts the likelihood of distant metastases, with higher-stage tumors at greatest risk. Furthermore, for rectal cancer, local stage also determines the surgical approach and whether preoperative chemoradiation should be administered. Traditionally, preoperative therapeutic response has been assessed by serial tumor size measurements [8, 9], and the need for postoperative adjuvant therapy has been determined by tumor stage. However, preclinical assessment of antiangiogenesis agents has highlighted limitations associated with standard morphologic measurements. In particular, response may be better assessed by alterations in vascular perfusion rather than size [10-12], and functional measurements may therefore be more appropriate.
Commercial CT software, based on differing mathematic analysis methods, including the deconvolution method and the slope method, is now available and provides quantitative perfusion measurements including tissue blood flow, blood volume, mean transit time, and capillary permeability. These CT measurements have recently been shown to change as a result of antiangiogenesis therapy [13]. Measurements using such software are assumed to be reliable [14-18], but at the time of writing, no study has specifically addressed inter- and intraobserver agreement of colorectal cancer measurements. Serial measurements must be accurate and reproducible, or clinical utility will inevitably be limited [19, 20]. In this study, we aimed to determine inter- and intraobserver variability for CT vascular perfusion measurements of primary colorectal cancer.
|
TopAbstractIntroductionSubjects and MethodsResultsDiscussionReferences |
|---|
CT
After patients had fasted 4 hr, 1,000 mL of water-soluble contrast
material, 2-4% meglumine and sodium diatrizoate (Gastrografin, Bracco), was
administered 30 min before CT to opacify the small bowel according to the
standard practice in our institution. The patient lay supine on the scanner
table, an 18-gauge venous cannula was placed in the antecubital fossa, and 20
mg of the spasmolytic hyoscine N-butylbromide (Buscopan, Boehringer
Ingelheim) was administered IV unless contraindicated. Buscopan was not
administered in four patients (13%). A restraining strap was placed around the
patient's abdomen to limit movement.
All patients were scanned using a 4-MDCT scanner (LightSpeed Plus, GE Healthcare). An abdominopelvic study was performed initially to identify the location of the tumor for planning purposes using the following parameters: slice thickness, 10 mm; interval, 5 mm; high-speed mode; speed, 30 mm/sec; pitch, 1.5; 120 kV; 180 mA; rotation speed, 0.6 sec; scan field of view (SFOV), 50 cm; and matrix, 512 x 512 mm. The images were then inspected on the CT console, the center of the tumor was identified, and the scan location was noted and used to plan the subsequent dynamic study.
For the dynamic study, a pump injector (Percupump Touch Screen, EZ-EM) was used to inject 100 mL of iopamidol (Niopam 340, Bracco) IV at a rate of 5 mL/sec. Four contiguous slices collimated to 5 mm each were obtained at 1-sec intervals through the center of the tumor using a cine mode (120 kV; 60 mA; SFOV, 50 cm; matrix, 512 x 512 mm). Acquisition commenced 5 sec after the start of IV injection and lasted a total of 65 sec. Patients were reminded to breathe gently during dynamic scan acquisition to minimize movement.
The dynamic study was followed immediately by a diagnostic portal venous phase abdominopelvic study. This was acquired 75 sec after IV contrast injection using the following parameters: slice thickness, 5 mm; interval, 2.5 mm; high-speed mode; 22.5 mm/sec; pitch, 1.5; 120 kV; 280 mA; rotation speed, 0.6 sec; SFOV, 50 cm; and matrix, 512 x 512 mm. This study was used to determine the local and distant stages of the tumor, and a radiologic report was issued according to usual practice.
Image Analysis
Image analysis was performed by two independent observers. Both were
radiologists and were unaware of each other's findings. All 31 perfusion
studies were analyzed on a stand-alone workstation using commercial
deconvolution-based CT perfusion software (Perfusion 2.0, GE Healthcare) and
the body tumor perfusion algorithm. Each individual perfusion study was loaded
into the software, and the single 5-mm slice that best depicted the tumor was
chosen from the four perfusion slices available. A 5-mm slice thickness was
selected because image noise from thinner collimations reduces the certainty
that the enhancement measurement is real and affects perfusion measurement. A
processing threshold of 0-120 H was selected so that the subsequent analysis
appropriately included soft tissue, both unenhanced and enhanced.
The arterial input was determined by placing a region of interest (ROI) over either the aorta or the iliac or superficial femoral artery, whichever was best visualized in the slice plane, and analyzing the attenuation change over the 65 sec of the perfusion acquisition. A time-attenuation curve was automatically generated for the arterial input along with perfusion maps for all the tissues within the scanning plane in the threshold selected. The individual perfusion maps generated were for blood volume, blood flow, mean transit time, and permeability.
An ROI was then drawn freehand around the peripheral margin of the tumor using an electronic cursor (Figs. 1A, and 1B). Care was taken to exclude perirectal and pericolonic fat and also intraluminal gas, a process that was facilitated by viewing a cine loop of the perfusion acquisition to gauge the degree and margins of patient movement during acquisition. A time-attenuation curve for the selected tumor tissue and the four perfusion parameters for the tumor tissue within the ROI was then derived. The mean values for the four tumor perfusion parameters for each individual patient were then noted by observers.
|
|
Statistical Analysis
Inter- and intraobserver agreement was assessed using two separate
analyses. First, intraclass correlation coefficients (95% confidence
intervals) were calculated for each of the four perfusion parameters (blood
volume, blood flow, mean transit time, and permeability) for each observer and
for both observers. Second, the mean differences, SD, and 95% limits of
agreement for each of the four perfusion parameters for each observer and for
both observers were calculated using the Bland-Altman test
[20]. The mean and SD for all
four perfusion parameters were calculated also for each observer.
|
TopAbstractIntroductionSubjects and MethodsResultsDiscussionReferences |
|---|
|
Interobserver Agreement
The intraclass correlation coefficients for the interobserver measurements
of each perfusion parameter are summarized in
Table 2. The intraclass
correlation coefficients ranged from 0.80 to 0.89, indicating apparently
excellent agreement. The mean difference, SD, and 95% limits of agreement for
interobserver measurements for each perfusion parameter are summarized in
Table 1, with corresponding
scatterplots that show the line of perfect agreement and Bland-Altman
agreement plots (Figs. 2A,
2B,
3A,
3B,
4A,
4B,
5A, and
5B). For all parameters
studied, the limits of agreement were more than the mean measurement.
Inspection of the graphs showed a clear tendency for observer 2 to make larger
measurements than observer 1 for all parameters studied. The results of the
95% limits of agreement analysis were less impressive than those for
intraclass correlation, raising the possibility of disagreement that might
potentially affect the clinical utility of these measurements.
|
|
|
|
|
|
|
|
|
Intraobserver Agreement
The intraclass correlation coefficient for intraobserver measurements for
each perfusion measurement is summarized in
Table 2. The intraclass
correlation ranged from 0.86 to 0.98 and again apparently indicated excellent
agreement. Intraobserver agreement was better than interobserver agreement for
all four perfusion measurements investigated. This was also reflected by
tighter 95% limits of agreement when compared with those obtained for the
interobserver measurements (Table
1). However, results of the 95% limits of agreement were similarly
less impressive than intraclass correlation analysis.
|
TopAbstractIntroductionSubjects and MethodsResultsDiscussionReferences |
|---|
Compared with other techniques, dynamic contrast-enhanced CT has the advantage of being widely available. The highly linear and predictable contrast pharmacodynamics of CT [21] are also cited as an attraction, with the clear implication that measurements will be reliable because of this. However, at the time of writing, this assumption is largely speculative in abdominal disease. Indeed, there are likely to be two potential sources of error: error due to inherent measurement variability attributable to the perfusion software or to inherent tumor heterogeneity (intrinsic error) and error due to observer differences (extrinsic error). For example, no two observers are likely to outline exactly the same ROI. For the technique to be clinically worthwhile, any differences attributable to these intrinsic and extrinsic errors must be small relative to the magnitude of changes expected as a result of therapy [20]. Most obviously, alteration in perfusion after antiangiogenesis therapy should be of an order of magnitude such that it swamps differences merely due to measurement error.
Although to date data relating to observer agreement for colorectal cancer perfusion measurement have not been published, Fiorella et al. [22] recently investigated the reproducibility of cerebral parenchymal perfusion measurements (cerebral blood volume, cerebral blood flow, and mean transit time) within an ROI placed by three observers. Intraclass correlation coefficients of 0.73, 0.87, and 0.89 were quoted for cerebral blood volume, cerebral blood flow, and mean transit time, respectively. The authors concluded that although there was a high degree of correlation between and within observers, the level of agreement was not sufficient to be used for clinical decision making; their findings are similar to our own.
Regarding agreement for other abdominal sites, a study of 10 patients used linear regression analysis to assess reproducibility of liver perfusion measurements and quoted an r2 value of 0.83 and 0.96 for the preportal liver gradient and arterially subtracted liver gradient, respectively [23]. Another study of splenic perfusion in 43 patients reported good interobserver agreement, citing a correlation coefficient (r) of 0.94 [24]. However, the use of simple linear correlation and regression to assess agreement is incorrect. Indeed, high correlation can be present when agreement is actually poor [19, 20, 25]. This is because correlation is merely a measurement of linear association. For example, if measurements obtained by one observer were exactly twice the value (or indeed half) of those obtained by another, the result would be perfect correlation because the plot of corresponding observer values would be a straight line [20]. Given this, the results described in the literature to date must be interpreted with caution.
As expected, we found that interobserver variability was greater than intraobserver variability, in concordance with other studies of observer agreement using traditional morphologic measures of response [26, 27]. Ostensibly, we found excellent correlations for the measurements performed for both inter- and intraobserver agreement. However, intraclass correlation, although more suitable to analysis of observer variation than simple correlation, splits the variability in the data into two components: that attributable to variability between observers and that attributable to variability between repeat measurements by the same observer. Therefore, we were not surprised to find less variability between two measurements by the same observer than between those from different observers. High correlation values are to be expected from this type of analysis and may imply good agreement when this level of agreement is actually insufficient for day-to-day clinical practice. Furthermore, this analysis examined only two observers and only two repeat measurements of each value. Because of this, the intraclass correlation coefficient should be interpreted together with the 95% limits of agreement.
We found relatively wide limits of agreement surrounding the mean measurement for all four perfusion parameters studied. This potentially limits application for assessment of therapeutic response, especially because sequential assessment of response is frequently made by different observers in day-to-day clinical practice. What was more surprising was the level of intraobserver variability, with limits of agreement that were still relatively wide for each of the four perfusion parameters. This will have implications for the use of CT perfusion imaging in monitoring therapeutic response, particularly if changes in perfusion parameters are small—for example, on the order of 10-20%. For example, if a drug is expected to reduce tumor blood flow only from 100 mL/100 g tissue/min to 80 mL/100 g tissue/min, it will be difficult to be certain whether measured change is due to therapeutic effect or measurement error on an individual patient basis given our limits of agreements.
At the time of writing, the expected changes in perfusion parameters contingent on antiangiogenesis therapy are uncertain. This is a novel and evolving field, and in many instances, our ability to obtain perfusion measurements is in advance of what is known about their behavior in human subjects. However, animal data suggest that changes in perfusion may be considerable. For example, a study of combretastatin A-4, an antivascular agent, was found to reduce blood flow in tumors in rats by 90% [28]. Whether the limits of agreement we found are acceptable for clinical practice should become clearer as these data from human subjects emerge.
Several factors contribute to measurement variability. Factors that are related to the individual who is making the measurements and that affect perfusion measurements include the positioning of the arterial and tumor ROIs. The software we used was based on deconvolution analysis, for which the definition of arterial input is essential. The arterial time-attenuation curve of the artery in the field of view is assumed to be the same as that of the vessel supplying the tumor, an assumption that is valid only in the absence of vascular stenosis or significant collateral circulation [14, 29]. Even so, minor differences in the shape of the arterial curve, especially the ascending slope, will affect the deconvolution process and perfusion values—in particular, the mean transit time and blood flow—and thus contribute to measurement variability [14].
Partial volume effects will influence interpretations. For example, if small vessels are used to obtain the arterial time-attenuation curve, movement as a result of vascular pulsation or respiration and laminar flow within the vessel will contribute to partial volume averaging that, in turn, affects measurements. In a study of cerebral perfusion in beagles, higher blood volume values were noted when smaller arteries were used for the input [14]. Although this can be compensated for by a correction algorithm, this is not available in the body tumor protocol used to measure perfusion in our study. More specifically with regard to bowel analysis, peristalsis will also contribute to partial volume effects, which we attempted to minimize using an antiperistaltic agent. The presence of air within the bowel lumen and fat surrounding the tumor will also both contribute to volume averaging effects within the tumor ROI, even if care is taken to exclude this, due to movement during respiration.
There will inevitably be some variation when drawing the ROI surrounding the tumor because the reviewer must trace this area freehand and must judge the margin between tumor and normal tissue subjectively. This is especially difficult when the tumor is ill defined, perhaps as a result of peritumoral inflammatory reaction or the presence of mural edema from associated ischemic colitis. Variability for repeated measurements could potentially be reduced if the observer was able to save an ROI for further use, a feature that may become available in the future.
The mathematic analysis method used by the software also will affect measurement agreement and possibly contribute to measurement variability. The deconvolution approach involves measuring the mean transit time of the system after an instantaneous arterial injection. An input residual function is derived from the arterial and tissue time-attenuation curves from which blood flow, blood volume, and mean transit time can be obtained [14]. This input residual function is not a unique mathematic value because of underlying noise. Thus, each time the software is used, perfusion measurements will differ even when the initial data, ROI, and operator are the same. Mathematic constraints are placed to limit this variation. Modification of the analysis method has also been necessary to take into account extravascular diffusion of contrast material within the tumor; a 13.2% variability was reported for blood flow when measurement was repeated in three rabbits under identical experimental conditions [15].
In summary, although we found that intraobserver agreement was better than interobserver agreement, the limits of agreement varied widely. As a consequence, observer agreement must be accounted for when using CT perfusion measurements to assess therapeutic response, especially if expected changes in perfusion are small. When practical, follow-up measurements on individual patients should be made by the original reviewer.
|
TopAbstractIntroductionSubjects and MethodsResultsDiscussionReferences |
|---|
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
This article has been cited by other articles:
|
|
 |
|
  G. Petralia, L. Preda, S. Raimondi, G. D'Andrea, P. Summers, G. Giugliano, F. Chiesa, and M. Bellomi Intra- and Interobserver Agreement and Impact of Arterial Input Selection in Perfusion CT Measurements Performed in Squamous Cell Carcinoma of the Upper Aerodigestive Tract AJNR Am. J. Neuroradiol., June 1, 2009; 30(6): 1107 - 1115. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 V. Goh, S. Halligan, F. Daley, D. M. Wellsted, T. Guenther, and C. I. Bartram Colorectal Tumor Vascularity: Quantitative Assessment with Multidetector CT--Do Tumor Perfusion Measurements Reflect Angiogenesis? Radiology, November 1, 2008; 249(2): 510 - 517. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
V. Goh, S. Halligan, A. Gharpuray, D. Wellsted, J. Sundin, and C. I. Bartram Quantitative Assessment of Colorectal Cancer Tumor Vascular Parameters by Using Perfusion CT: Influence of Tumor Region of Interest Radiology, June 1, 2008; 247(3): 726 - 732. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. C. Faria, C. S. Ng, K. R. Hess, S. Phongkitkarun, J. Szejnfeld, D. Daliani, and C. Charnsangavej CT Quantification of Effects of Thalidomide in Patients with Metastatic Renal Cell Carcinoma Am. J. Roentgenol., August 1, 2007; 189(2): 378 - 385. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. Hakime, H. Peddi, A. U. Hines-Peralta, C. J. Wilcox, J. Kruskal, S. Lin, T. de Baere, V. D. Raptopoulos, and S. N. Goldberg CT Perfusion for Determination of Pharmacologically Mediated Blood Flow Changes in an Animal Tumor Model Radiology, June 1, 2007; 243(3): 712 - 719. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
V. Goh, S. Halligan, and C. I. Bartram Quantitative Tumor Perfusion Assessment with Multidetector CT: Are Measurements from Two Commercial Software Packages Interchangeable? Radiology, March 1, 2007; 242(3): 777 - 782. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
V. Goh, S. Halligan, S. A. Taylor, D. Burling, P. Bassett, and C. I. Bartram Differentiation between Diverticulitis and Colorectal Cancer: Quantitative CT Perfusion Measurements versus Morphologic Criteria--Initial Experience Radiology, February 1, 2007; 242(2): 456 - 462. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 V Goh, S Halligan, L Gartner, P Bassett, and C I Bartram Quantitative colorectal cancer perfusion measurement by multidetector-row CT: does greater tumour coverage improve measurement reproducibility? Br. J. Radiol., July 1, 2006; 79(943): 578 - 583. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
V. Goh, S. Halligan, J.-A. Hugill, and C. I. Bartram Quantitative assessment of tissue perfusion using MDCT: comparison of colorectal cancer and skeletal muscle measurement reproducibility. Am. J. Roentgenol., July 1, 2006; 187(1): 164 - 169. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
Q.-S. Ng, V. Goh, H. Fichte, E. Klotz, P. Fernie, M. I. Saunders, P. J. Hoskin, and A. R. Padhani Lung Cancer Perfusion at Multi-Detector Row CT: Reproducibility of Whole Tumor Quantitative Measurements Radiology, May 1, 2006; 239(2): 547 - 553. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. Sahani, S. Kalva, and P. F. Hahn Response Radiology, February 1, 2006; 238(2): 757 - 757. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
