| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
1 Department of Radiology, Memorial Sloan-Kettering Cancer Center, 1275 York
Ave., New York, NY 10021.
2 Weill Medical College of Cornell University, 1300 York Ave., New York, NY
10021.
Received April 14, 2003;
accepted after revision July 24, 2003.
Presented at the annual meeting of the American Roentgen Ray Society, San
Diego, CA, 2003.
Abstract
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
MATERIALS AND METHODS. Written interpretations made by on-call fellows were compared with reports approved by attending radiologists for all CT angiograms obtained for suspected pulmonary embolism after work hours and on weekends in a recent 19-month period. Interpretations were stratified as positive, negative, or equivocal for pulmonary embolism. In cases of discordant interpretations, those CT angiograms were rereviewed by two thoracic radiologists; then patient medical records were reviewed for evidence of clinical effect. Technical and patient-related reasons for discordant interpretations of CT angiograms were recorded.
RESULTS. Six hundred fifty-eight oncology patients were examined on
CT angiography; five were examined twice. The fellows reported 137 CT
angiograms (21%) as positive, 498 (75%) as negative, and 28 (4%) as equivocal
for pulmonary embolism. Interpretations of the fellows and attending
radiologists agreed in 93% (615/663) of CT angiograms (
= 0.80). The
concordance rates for CT angiograms interpreted by fellows as positive (89%,
122/137), negative (96%, 479/498), and equivocal (50%, 14/28) were
significantly different from each other (p < 0.001 for each). A
significantly greater proportion of CT angiograms with discordant
interpretations was reported to be technically limited (p < 0.01).
No clear adverse clinical events were attributed to discordant interpretations
of CT angiograms, although the death of one patient in that subgroup was of
indeterminate cause.
CONCLUSION. In the evaluation of CT angiograms obtained for suspected pulmonary embolism, on-call fellows showed good agreement with attending radiologists. CT angiograms with discordant interpretations often were limited by technical or patient-related factors.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
= 0.85)
[2,
14]. In our institution, which is a dedicated tertiary care cancer center, radiology fellows are responsible for interpreting all emergent CT examinations performed after routine work hours. The objective of this retrospective study was to evaluate interobserver variability between interpretations by our on-call radiology fellows and subsequent interpretations by attending radiologists of CT angiograms for the diagnosis of pulmonary embolus, factors contributing to discordance in interpretations, and the clinical relevance of discrepancies in a high-risk oncology population.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
CT Angiography Technique and Interpretation
Axial CT scans of the chest were obtained with a four-row MDCT scanner
(Lightspeed, General Electric Medical Systems, Milwaukee, WI) in 640 CT
angiograms. Four-hundred fifty-seven CT angiograms were obtained with 2.5-mm
collimation and no overlap, at 7.5 mm per rotation, acquired in a single
breath-hold from 2 cm below the domes of the diaphragm to the top of the
aortic arch. Images were reconstructed at 1.25-mm slice thickness at 1-mm
intervals with the standard algorithm. A total of 140 mL of nonionic contrast
material was administered at an injection rate of 4 mL/sec with a 25-sec
injection-to-scan delay. One-hundred eighty-three CT angiograms were obtained
with the same parameters, but no thinner or overlapping reconstructions were
performed. Twenty-three other CT angiograms were obtained using a
single-detector helical scanner (HiSpeed; General Electric Medical Systems)
with 3-mm collimation and no overlap at a pitch of 1.6; these images were
reconstructed at 3-mm slice thicknesses at 1-mm intervals with the standard
algorithm. Images were acquired in a single breath-hold from 2 cm below the
domes of the diaphragm to the top of the aortic arch. A total of 150 mL of
nonionic contrast material was administered at an injection rate of 4 mL/sec
with a 30-sec injection-to-scan delay.
All 663 CT angiograms were initially interpreted on a PACS (picture archiving and communication system) workstation by one of 31 (1-year) body imaging fellows who recorded their CT angiogram interpretations in a CT log book while on call. Each CT angiogram was reviewed by one of 24 attending radiologists on our body CT service the next morning before a final report was issued for the CT angiogram. The preliminary interpretation provided by the on-call fellow was available to the attending radiologist. Standard CT angiography diagnostic criteria for pulmonary embolus were used, including presence of an intraluminal filling defect (typically on more than one contiguous axial section) with possible associated expansion of the vessel and abrupt termination of an opacified vessel peripherally. Reconstruction of image data in nonaxial planes or in three dimensions was not performed in any case.
Data Collection and Analysis
For this study, the fellows' interpretations were retrieved from the CT log
book (in which preliminary interpretations of all emergent off-hours CT scans
are recorded by the on-call fellow), and final reports of attending
radiologists were reviewed in the radiology information system. Our
institutional review board granted approval for this retrospective study.
Interpretations of CT angiography by fellows and attending radiologists were each stratified as positive, negative, or equivocal (i.e., possibly positive or uninterpretable for technical reasons) for pulmonary embolus. Factors that were reported to limit the quality of a CT angiogram (such as poor IV contrast bolus, respiratory motion, streak artifact, or adjacent pulmonary abnormality or adenopathy) were recorded. Results of subsequent radiology studies such as a repeated CT angiogram, extremity sonogram, or nuclear ventilation–perfusion scan that were obtained within 2 days after the initial CT angiogram were recorded. The number of years of postfellowship experience of the attending radiologist interpreting each CT angiogram was stratified as 1–3, 3–5, 5–10, 10–20, and more than 20 years. CT angiograms have been used routinely for evaluating suspected pulmonary embolus for only approximately the last 4 years in our institution; none of our attending radiologists had more than those 4 years' experience with CT angiography for pulmonary embolus.
In each case of discordant interpretation only, the CT angiogram was reviewed in consensus on a workstation by two faculty radiologists (neither of whom was involved in the discordant interpretation) with 6 and 19 years' experience in both chest and body CT imaging, respectively. These radiologists classified each discordant case as positive, negative, or equivocal; the pulmonary emboli in each case considered positive by these radiologists were also classified as being located in central pulmonary arteries (main, right, and left) or in segmental–subsegmental pulmonary artery branches. The electronic medical records were subsequently reviewed to evaluate any adverse effect on patient management of the discordant interpretation of the CT angiogram.
Percentage of agreement was determined for overall agreement and for each of the three groups of interpretations of CT angiograms. The chi-square test was used to show statistically significant differences between proportions in the discordant and concordant interpretation groups. The kappa coefficient also was used to assess agreement. Kappa values were interpreted as less than 0.20, poor; 0.21–0.40, fair; 0.41–0.60, moderate; 0.61–0.80, good; and 0.81–1.0, very good [15].
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
= 0.80); the agreement was 89%
(122/137) for CT angiograms interpreted as positive by the on-call fellow, 96%
(479/498) for CT angiograms interpreted as negative, and 50% (14/28) for CT
angiograms interpreted as equivocal (Table
1).
|
The agreement for CT angiograms interpreted by the on-call fellow as
negative (96%) was significantly higher than that for CT angiograms
interpreted as positive (89%), and the percentages of agreement for positive
and negative CT angiograms were each significantly higher than those for
equivocal CT angiograms (50%) (p < 0.001 for each). The agreement
between fellows and attending radiologists was stronger for CT angiograms
interpreted by the fellow as positive versus those interpreted as negative
( = 0.90) than for CT angiograms interpreted as either positive or
negative versus equivocal ( = 0.77 and 0.55, respectively).
Concordance Rate by Quality of CT Angiogram
Overall, quality was reported to be suboptimal in 11% (71/663) of CT
angiograms because of technical or patient-related factors. Only 9% (54/615)
of CT angiograms with concordant interpretations were reported to be limited,
compared with 35% (17/48) of CT angiograms with discordant interpretations
(p < 0.01). Among those CT angiograms with limitations, no
significant difference was evident in the distribution of factors such as poor
contrast bolus, respiratory motion, streak artifact, or adjacent abnormalities
between the CT angiograms with concordant and those with discordant
interpretations (p > 0.05); comparison is limited, however, by the
small number of CT angiograms in each of those subcategories. In addition, no
association was shown between slice thickness and rate of agreement
(p > 0.05).
Rereview of CT Angiograms with Discordant Interpretations
The two study radiologists disagreed with interpretations of the initial
attending radiologists in 25% (12/48) of discrepant interpretations of CT
angiograms. The discordant interpretation involved a segmental or subsegmental
pulmonary artery branch in 46 (96%) of the 48 studies (Figs.
1 and
2). In two CT angiograms, the
discrepant interpretation involved findings in the right main pulmonary artery
(Fig. 3A,
3B). In 10 CT angiograms, the
two study radiologists agreed with the interpretation of the on-call fellow,
and in two CT angiograms, they disagreed with the interpretations of both the
fellow and the attending radiologist. The initial 7% (48/663) disagreement
rate between on-call fellows and attending radiologists in interpreting CT
angiograms would decrease to 5% (36/663) if these 10 CT angiograms were
considered as having been correctly interpreted by the on-call fellow.
|
|
|
|
Level of Experience of Attending Radiologists in Discordant Interpretations of CT Angiograms
Of the 24 different attending radiologists who interpreted these CT
angiograms, six had 1–3 years' posttraining experience, two had
3–5 years, seven had 5–10 years, three had 10–20 years, and
six had more than 20 years. More disagreement with the fellows' original
interpretations was noted among faculty with more than 5 years' posttraining
experience; 90% (43/48 CT angiograms) of the disagreement occurred in that
group. Disagreement was noted in 5% (5/97) of the CT angiograms interpreted by
attending radiologists with 1–3 years' experience, in 0% (0/34)
interpreted by attending radiologists with 3–5 years' experience, in 8%
(19/225) interpreted by attending radiologists with 5–10 years'
experience, in 10% (13/124) interpreted by attending radiologists with
10–20 years' experience, and in 6% (11/183) interpreted by attending
radiologists with more than 20 years' experience. The two study radiologists
who rereviewed those CT angiograms with discordant interpretations agreed with
the initial interpretation of the attending radiologist in five of five, zero
of zero, 14 of 19, nine of 13, and eight of 11 CT angiograms,
respectively.
Clinical Outcome
Table 2 summarizes the types
and frequencies of discordant interpretations, the type of subsequent
radiology examination performed, whether anticoagulation therapy was
administered, and whether management changed after a discordant interpretation
of a CT angiogram. In most cases of discordance, treatment was altered to be
consistent with the interpretation of the attending radiologist. No apparent
adverse clinical effect was clearly attributed to discordant interpretations
in this study, although the death of one patient 1 day after CT angiography
was performed was of indeterminate cause (and conceivably could have been due
to pulmonary embolus, although the on-call fellow interpreted the CT angiogram
as positive and the attending radiologist interpreted it as equivocal). In
patients whose CT angiograms were initially interpreted as positive by the
on-call fellow and subsequently interpreted as negative by the attending
radiologist, no clinically evident hemorrhages were seen within 48 hr after
the institution of anticoagulation.
|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
In our institution, fellows—not residents—provide all
preliminary interpretations of off-hours emergency CT scans. Our retrospective
study revealed a high overall concordance (93%; = 0.80) between
interpretations of CT angiograms by fellows and attending radiologists; this
concordance would increase to 95% if the 10 CT angiograms in which the
fellows' initial interpretations were deemed correct by the reinterpretations
of the two study radiologists were considered concordant. Also, Mayo et al.
[2] and Chartrand-Lefebvre et
al. [14] reported
interobserver agreement among attending radiologists for interpretation of
pulmonary CT angiograms to be very good (range of agreement, 83–88%;
= 0.85); our results are quite similar to theirs.
Interpretations by fellows have not been evaluated to the same extent as those of residents in prior studies, likely because at many institutions residents are responsible for providing on-call interpretations of emergency studies; fellows serve as backup call or junior faculty because most fellows are board-certified in radiology and are more experienced than residents. Fellows who are board-eligible or board-certified would be expected to have a rather high agreement with the faculty. Perfect concordance among radiologists—including the faculty—would not be seen with any test, regardless of level of training, because of the inherent subjectivity of radiologic interpretation and resulting interobserver variability.
Faculty with more than 5 years of posttraining experience were more likely to disagree with the on-call fellow's interpretation than were junior faculty. Perhaps midlevel and senior faculty are more confident in questioning a fellow's interpretation than are faculty who only recently were fellows themselves. Also, given that the preliminary interpretation provided by the on-call fellow was available to the attending radiologist, it is possible that a more junior attending radiologist might be strongly influenced by that preliminary interpretation and thus be less likely to disagree with the fellow's interpretation. Disagreement between interpretations of attending radiologists and the two study radiologists was noted in 25% of the discordant interpretations of CT angiograms, indicating that those cases likely were particularly difficult to interpret, even for faculty. In 35% of discordant interpretations, the CT angiograms were limited by one or more factors such as adjacent pulmonary parenchymal opacities or adenopathy, respiratory motion, poor IV contrast bolus, or streak artifact.
Our study was not designed to measure the accuracy of CT angiography, the relative performance of fellows and attending radiologists, the interobserver variability among attending radiologists, or the learning curve for CT angiography, but rather to evaluate the interobserver agreement between radiology fellows and attending radiologists in interpretation of CT angiograms. Interobserver agreement among attending radiologists for interpretation of pulmonary CT angiograms previously has been shown to be very good [2, 14]. Also, the intent of our study was to assess our actual clinical practice situation, rather than a researchmode interpretation of each CT angiogram by thoracic radiologists.
In our study, the CT angiogram was interpreted by the fellow as positive in 21% of studies. This somewhat high positive rate is likely due to the fact that all patients in this study were at increased risk for pulmonary embolus due to their having cancer.
We recognize that our study has several limitations. Our data were obtained retrospectively from a CT log book as recorded by the fellow while interpreting the CT angiogram on call, as well as from the report of the interpretation of attending radiologists the following day, rather than from a rereview of all the actual CT angiograms. When these CT angiograms were interpreted, neither the fellow nor the attending radiologist knew that their interpretations would be part of a research project; thus, our results reflect actual clinical practice in an academic setting. Also, some preliminary reports may not have been recorded in the log book and thus would not have been included in this study; however, this event is uncommon in our department because the attending radiologists actively seeks these preliminary interpretations.
Fellows are not a homogeneous group, having had various amounts of experience in interpreting CT angiograms during their residency training; therefore, the results of our study might differ if a different group of fellows had been involved. A fellow is not a static entity. During the first few days of fellowship, fellows on call may have little more experience with CT angiography for pulmonary embolus than they had as residents a few days earlier; but by the end of the fellowship year, fellows will have learned to interpret CT angiograms in a manner similar to that of their supervising attending radiologists. We did not stratify discrepancy rates by the relative time of the fellowship year, which might have shown improved concordance over the course of a year; however, the number of CT angiograms interpreted by any one fellow was insufficient to allow meaningful statistical analysis.
Determining the downstream clinical effect of changes in on-call interpretations of CT angiogram in retrospect is difficult; however, any major adverse effects would usually be apparent from a review of a patient's medical records, as was performed in this study. The clinical significance of a delay in diagnosis of pulmonary embolus for 18 hr or less is debatable in most patients, although it might be critical in patients with limited cardiopulmonary reserve or other concurrent severe medical illnesses. Although no adverse clinical effect of discordant interpretations was proven in this study, such effects may be seen if a larger number of CT angiograms were included. In the one patient with a discrepant interpretation of a CT angiogram who died, the CT angiogram was actually interpreted as positive by the on-call fellow and as equivocal by the attending radiologist; the clinical impact of this discrepancy is unclear. Also, results of this study performed in oncology patients may not be generalizable to other patient populations.
In summary, we found a high overall concordance between interpretations of radiology fellows and attending radiologists of CT angiograms obtained in a high-risk oncology population, particularly for CT angiograms interpreted by the fellow as positive or negative (vs those considered to have equivocal results). Discordance often occurred in cases that had technical limitations or concomitant thoracic abnormalities. No clear adverse patient events were attributed to discordant interpretations in this study.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
This article has been cited by other articles:
|
|
 |
|
  V. King, A. A. Vaze, C. S. Moskowitz, L. J. Smith, and M. S. Ginsberg D-Dimer Assay to Exclude Pulmonary Embolism in High-Risk Oncologic Population: Correlation with CT Pulmonary Angiography in an Urgent Care Setting Radiology, June 1, 2008; 247(3): 854 - 861. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. E. Jones and C. Wittram The Indeterminate CT Pulmonary Angiogram: Imaging Characteristics and Patient Clinical Outcome Radiology, October 1, 2005; 237(1): 329 - 337. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
