| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
1 All authors: Department of Radiology, Robert Wood Johnson University Hospital, 1 Robert Wood Johnson Pl., MEB 4th Fl., New Brunswick, NJ 08901.
Received December 2, 2002;
accepted after revision January 22, 2003.
Address correspondence to E. Carney.
Abstract
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
MATERIALS AND METHODS. Any discrepancies between the preliminary and final interpretations were judged as either major (i.e., necessitating an urgent change in treatment) or minor errors. We conducted patient follow-up via a retrospective review of the medical charts to determine whether any of the discrepancies led to additional imaging, an increase in patient morbidity, an extension of a hospital stay, or a change in treatment.
RESULTS. The overall discrepancy rate in interpretations rendered by the residents and those performed by the attending radiologist was 3.8%, with most of these discrepancies (86%) judged to be minor. If we combined the data for body CT scans and sonograms, the rate of minor discrepancies was 3.2%, and the rate of major discrepancies was 0.5%. If we considered only body CT data in the evaluation, the overall discrepancy rate increased to 6.4%, with a 5.4% rate of minor discrepancies and a 1.0% rate of major discrepancies.
CONCLUSION. Our evaluation of discrepancy rates was unusual in that we included interpretations of sonograms, on which residents and the attending radiologist had a higher rate of agreement (99.5%). Because of the high agreement in the interpretation of sonograms, the overall discrepancy rate was 3.8%. However, if only body CT scan interpretations were evaluated, our results were closer to the rates reported in previously published studies. Major discrepancies led to a change in patient treatment but did not lead to any increase in patient morbidity or to any quantifiable increase in the length of the hospital stay.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Because the significance of a misinterpretation is ultimately determined by its impact on patient care, we evaluated whether the discrepancies resulted in any change in immediate patient care, specifically in the need for additional imaging, an increase in patient morbidity, or extension of the hospital stay.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Imaging was performed with a HiSpeed Enhancer CT scanner (General Electric Medical Systems, Milwaukee, WI) and an HDI-5000 sonography unit (ATL Ultrasound, Bothell, WA). The CT protocol was to acquire contiguous axial 3-mm sections through the chest with a pitch of 3.0 and contiguous axial 7-mm sections through the abdomen and pelvis with a pitch of 1.5. All sonograms were obtained by experienced sonographic technologists, with on-call residents performing additional scanning in approximately 10% of the cases. The length of experience of the technologists ranged from 4 to 33 years.
All discrepancies discovered during this period were judged to be either minor or major, depending on whether any urgent change in treatment was required after the discrepant finding was reported or whether the discrepancy resulted in a change in the clinical status of the patient. We made this determination after retrospectively reviewing the charts of patients who had discrepancies between the preliminary and final radiologic interpretations. A major discrepancy was defined as one resulting in increased patient morbidity, a change in treatment, additional imaging, or an extension of the hospital stay. Imaging findings deemed to be major discrepancies were subsequently reviewed a second time by the body imaging specialist and also by two diagnostic radiologists. Interobserver agreement on these findings was then evaluated.
For the final 2 months of the study, we did not record discrepancies involving small, incidentally discovered pulmonary nodules; liver lesions; or presumed renal cysts from the database used to record study results because these conditions had no impact on acute patient care and were presumed to be irrelevant findings. However, these missed findings were reviewed with the residents the next morning and reported to the appropriate clinical service. The proportion of the total number of minor discrepancies was extrapolated from the data collected during the first 4 months of the study. Major discrepancies were recorded during the entire 6 months. We determined statistical significance using chi-square methodology to detect any correlation between the discrepancy rate and the level of resident training.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
|
Five of the discrepancies were considered major because the final interpretations required an urgent change in patient treatment. These errors included two cases of missed pulmonary emboli and one case each of early appendicitis, mediastinal hematoma, and innominate artery occlusion. In each of these cases, emergency alteration in patient care was required after the discrepant findings were reported.
In the case of the initially missed appendicitis, the patient had undergone CT because appendicitis was suspected; the CT scan was interpreted by the resident as showing negative findings. The patient was discharged from the emergency department with a tentative diagnosis of gastritis or peptic ulcer. Findings suggestive of early appendicitis were noted later by the attending radiologist (Fig. 1). The emergency department was notified that morning of the abnormal findings. The patient was called back to the hospital for a surgical consultation and subsequently underwent an uncomplicated appendectomy later that day.
|
An innominate artery occlusion was not identified by the resident initially interpreting a CT scan but was noted by the attending radiologist the following morning (Fig. 2A, 2B). The patient with the occlusion subsequently underwent MR angiography of the aortic arch and the subclavian and carotid arteries and aortography of the aortic arch. Both imaging studies showed a 90–95% occlusion of the innominate artery. The patient underwent an innominate artery endarterectomy 2 days later. Although this case was a major discrepancy, the initial misinterpretation had no effect on patient morbidity or the length of the hospital stay.
|
|
The case of the initially missed anterior mediastinal hematoma involved a trauma patient who was brought to the emergency department after a motor vehicle crash. The patient had sustained multiple injuries, including pulmonary contusions and splenic injury. The CT scan showed a soft-tissue density in the retrosternal anterior mediastinum extending along the pulmonary trunk and anteriorly to the ascending aorta. This finding is consistent with the presence of blood in the mediastinum. It was missed by the resident on-call but was noted by the attending radiologist (Fig. 3). The mediastinal hematoma was reported to the appropriate physician the next morning, and the patient immediately underwent thoracic aortography. Findings of the aortogram were negative, and the patient had no increased morbidity because of the error in the initial interpretation.
|
The fourth case involved a patient who presented with a 3-week history of mild chest pain that was aggravated by respiration. The patient reported experiencing an increase in pain on the day of presentation. A CT pulmonary arteriogram obtained in the patient was interpreted by the resident as showing negative findings; the resident also interpreted a bilateral lower extremity venous duplex sonogram as showing negative findings and a ventilation–perfusion scan as showing the patient had a low probability of disease. The patient was sent home with a diagnosis of atypical chest pain. The next morning, the attending radiologist identified two small subsegmental pulmonary emboli on the CT pulmonary arteriogram. The patient was then called back to the emergency department for a second ventilation–perfusion scan, which was again interpreted as showing a low probability of disease, and a second bilateral lower extremity venous duplex sonogram, the findings of which were again interpreted as negative. The patient was judged to have no need for anticoagulation therapy and was subsequently sent home with instructions to follow up with his primary care doctor. Although this patient had no increased morbidity and required no change of treatment, he was required to undergo the repeated studies the next morning.
The fifth patient with initially misinterpreted findings also had a pulmonary embolus. The patient was an 83-year-old man with a history of bladder cancer who presented with dyspnea and fever. Pulmonary embolism was clinically suspected, so the patient underwent multidetector CT, the results of which were originally interpreted as negative by the resident. In addition, a duplex sonogram revealed deep venous thrombosis, so the patient was placed on a short-term course of anticoagulation. A few hours later, the attending radiologist made the diagnosis of pulmonary embolism, and the patient's physicians were notified of the discrepancy. Long-term anticoagulation was contraindicated in light of a history of hematuria, and the patient was immediately scheduled for placement of an inferior vena cava filter. The discrepancy in the interpretations, therefore, resulted in a change in patient treatment.
The overall rate of minor errors in interpretations was 3.2%, a percentage that reflects all minor discrepancies that occurred during the first 4 months of the study. During the final 2 months of the study, discrepancies that were not clinical emergencies, such as incidentally discovered pulmonary nodules, were not recorded. By extrapolating from the data collected the first 4 months, we determined that approximately seven additional minor discrepancies in interpretations of body CT findings probably occurred during the final 2 months of the study. Minor errors often included incidentally discovered liver lesions, pulmonary nodules, and small hematomas judged to be clinically insignificant. No additional imaging and no change in subsequent treatment were required for these patients because the initial images were misinterpreted.
We found that 13 discrepancies (37%) in interpretations involved first-year
residents, 12 (34%) involved second-year residents, and 10 (29%) involved
third-year residents. During the time of the study, fourth-year residents were
not on call. In correlating the rate of errors with the number of years of
post-graduate training that the residents had, we found that the distribution
was statistically insignificant (p < 0.01) between different
levels of training (
22 = 0.348). This
analysis is of limited value because of the small number of cases. Most of the
errors were false-negative findings (i.e., failure to recognize disease or an
abnormal finding). The one false-positive case involved a congenital variant
of a transverse process that was interpreted as a fracture. The error did not
affect patient care.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
In our study, the overall discrepancy rate between the interpretations by the radiology residents and the attending radiologists was 3.8% for body CT and sonographic examinations. To our knowledge, ours is the first study to evaluate discrepancies between the after-hours interpretations of sonograms by residents and the final interpretations by the attending radiologists. We found few discrepancies (two cases among interpretations of 415 sonograms) between the residents' and the attending radiologist's interpretations of the sonograms. Because the quality of sonograms is operator-dependent, our low rate of errors is in part a reflection of the skill and training of the sonographic technologists performing these examinations.
Our results for resident errors in the interpretation of CT scans are similar to those found in other studies [1–5]—a minor discrepancy rate of 5.4% and a major discrepancy rate of 1.0% (Table 2). We find it notable that our finding of a 1.0% rate of major discrepancies in the interpretation of body CT scans is the lowest reported in the literature thus far. When combining findings for CT and sonographic examinations, our discrepancy rates decrease to 3.2% for minor errors and 0.5% for major errors.
|
Other studies have found comparable discrepancy rates between residents and staff radiologists in interpretations of after-hours emergency CT scans. For example, Velmahos et al. [1] evaluated discrepancies between CT interpretations by residents and attending general radiologists. Over a 6-month period, they found a discrepancy rate of 11% between the interpretations by the residents and those by the attending radiologists, 5% of which were significant. Similarly, in a study of discrepancies between interpretations of off-hours emergency CT scans by residents and a single attending body CT radiologist, Wechsler et al. [2] reported an overall discrepancy rate of 7.7%, 1.2% of which were major discrepancies and the remainder of which were minor discrepancies. Yoon et al. [3] compared interpretations of abdominal and pelvic CT scans obtained in trauma patients made by attending general radiologists and subspecialty imaging radiologists and found a 29.9% rate of overall discrepancies, with a major discrepancy rate of 2.3%. Roszler et al. [4] evaluated discrepancies between residents and a panel of three attending general radiologists in the interpretation of after-hours emergency cranial CT scans and found a 2% rate of moderate or major discrepancies. Wysoki et al. [5] assessed the discrepancy rate between interpretations made by on-call radiology residents and those made by attending neuroradiologists of posttraumatic cranial CT scans and found a 1.7% rate of major discrepancies and 2.6% rate of minor discrepancies.
Nearly all (97% or 34/35) the errors made in our study were false-negative findings. This result is consistent with prior studies showing false-negative detection errors occur more often than false-positive detection errors. The number of minor discrepancies in our study was more than six times that of major discrepancies. A prior study has shown that the percentage of errors increases as the importance of the error decreases [2]. Clinically insignificant findings—those that are of less immediate concern—have an increased probability of being overlooked.
The method of providing after-hours coverage in radiology departments varies from institution to institution. In academic departments with residency programs, residents perform this valuable service with attending radiologists in-house or on-call. In actual practice, not all attending radiologists who interpret emergency CT scans and sonograms are body imaging specialists; they often are general diagnostic radiologists or subspecialists in other areas.
Adopting the body imaging specialist as the gold standard for accuracy introduces several weaknesses into our analysis, the first of which is the assumption that the final interpretation of one individual specialist is correct. For example, Yoon et al. [3] compared interpretations of abdominal and pelvic CT scans made by general radiologists with those that were made by subspecialty abdominal imaging radiologists as part of a quality assurance program. In one third of the cases involving additional diagnostic images needed for reevaluation, the interpretations made by the quality assurance reviewers were rejected in favor of the interpretations of the general radiologists. Another weakness is the assumption that if residents were not offering preliminary interpretations of these examinations, the interpretations would be made by body imaging specialists. In fact, the likelihood is that the emergency imaging examinations would just as likely be interpreted by a general diagnostic radiologist or a subspecialist in another area, a circumstance that potentially carries an error rate greater than that with a body imaging specialist. Alternatively, emergency imaging interpretation could be performed from a remote site by an attending radiologist using teleradiology. When examined, the accuracy achieved with this paradigm was shown to produce results that were no different than the accuracy achieved with on-site interpretation by radiologists [6].
Two of the major discrepancies in our study involved pulmonary emboli that were identifiable on CT pulmonary arteriograms. This finding is not surprising because significant interobserver variability has been reported in the interpretation of CT pulmonary angiograms. Sostman et al. [7] reported a sensitivity ranging from 62% to 92% among the attending radiologists interpreting CT pulmonary angiograms, which suggests that interobserver variability may be a potentially important limitation of CT pulmonary angiography that is unrelated to the interpreter's level of training. In our institution, only one of three attending radiologists, the body imaging specialist, was able to identify the subsegmental pulmonary emboli, which tends to confirm that there is a high level of interobserver variability. The two other (diagnostic) radiologists were not able to locate them. This result suggests the possibility of overinterpretation by the body imaging specialist or a failure by the residents and other attending radiologists to observe imaging findings.
CT pulmonary angiography tends to be more reliable and sensitive in revealing emboli in the main pulmonary arteries or the first branches of these arteries [8, 9]. Sensitivity and specificity significantly decrease for revealing emboli in the smaller segmental and subsegmental branches. In one study [10], sensitivity of CT pulmonary angiography dropped from 86% for revealing emboli in central pulmonary arteries to 63% when peripheral branches were included. For this reason, the interpreter's level of training may not have had a significant impact on the discrepant interpretations.
One of our major discrepancies involved a missed case of early appendicitis. The primary CT criteria for diagnosis of acute appendicitis were identified in a previous study [11] as an appendix with a transverse diameter exceeding 6 mm and associated periappendiceal inflammatory changes. In our study, the CT scan obtained in the patient with the missed case of appendicitis showed a thickened appendiceal wall with a transverse diameter of 1 cm and slight streaking that represented mild periappendiceal inflammatory changes. Various studies [11–13] have reported the diagnostic accuracy of CT in patients with suspected appendicitis. With attending radiologists interpreting CT scans, sensitivities ranging from 87% to 98% and specificities ranging from 83% to 97% have been reported. Studies [14, 15] have shown the rate of agreement between residents and attending radiologists in the interpretation of CT scans obtained for suspected appendicitis to be 91–93%. In a study by Lowe et al. [15] of discrepancies between residents' and attending radiologists' interpretations of CT scans obtained for suspected appendicitis, the residents achieved a sensitivity of 63%, specificity of 96%, and accuracy of 88%, whereas the attending radiologists achieved a sensitivity of 95%, specificity of 98%, and accuracy of 97%. In our institution, approximately 6% of cases of appendicitis are found on CT scans obtained in emergency patients. Our residents, therefore, were estimated to have identified appendicitis in approximately 98% of these cases during the 6 months of the study
In our study, the body imaging specialist was the only attending radiologist who rendered the final interpretation of the images. Although we believe that having a specialist render the interpretations is one of the strengths of our study, it is also a potential limitation because we made the interpretation by one individual the gold standard. However, when all five cases of major interpretation discrepancy were reviewed by the body imaging specialist and two diagnostic radiologists, all three radiologists agreed on four of the five major discrepancies; the two diagnostic radiologists were not able to locate the pulmonary emboli on the CT scan of one of the patients.
Another potential limitation of our study involves the fact that the studies were not performed consecutively. As a consequence of having the interpretation of only one specialist serve as the gold standard, we were not able to have every CT scan or sonogram obtained during the 6-month period interpreted by this same attending radiologist. The imaging studies were distributed fairly uniformly over the 6-month period but were not acquired consecutively. In addition, because of the small number of major findings, correlation between discrepant interpretations and the level of resident training could not adequately be determined. Further clinical data collection is needed for a more accurate determination of any correlation between the two factors.
Previous studies comparing body imaging interpretations made by residents with those made by attending radiologists have reported no adverse clinical outcomes as a result of the discordant findings [1, 2]. Similarly, none of the interpretation discrepancies in our study led to increased patient morbidity or mortality. Most of the minor discrepancies were not relevant to acute patient care and had little effect on urgent patient treatment. Through patient follow-up and chart review, we were able to determine that the major interpretation discrepancies did not lead to any increase in patient morbidity. However, additional imaging and changes in treatment were necessary in these cases.
In conclusion, rates of both the major and minor discrepancies in interpretation of emergency CT scans and sonograms between radiology residents and attending radiologists are low when residents are well trained and have progressed beyond the basic level of learning. Radiology trainees tend to be more likely to miss abnormal findings than to interpret images of normal findings as abnormal.
We found a high (99.5%) rate of agreement in the interpretations of emergency sonograms between the residents and the attending radiologist. If only the discrepancies in interpretations of body CT scans are evaluated, the discrepancy rate increases to 6.4%, a rate that more closely approximates those reported by other (previously cited) studies. In our study, patient care was not adversely affected by having the radiology residents interpret emergency CT and sonographic images after-hours and the attending radiologist review the interpretations the next morning. It may be advisable for other academic radiology departments to perform a similar internal review of residents' after-hours performance as part of quality assurance of the departmental training program.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
This article has been cited by other articles:
|
|
 |
|
  C. G. Filippi, B. Schneider, H. N. Burbank, G. F. Alsofrom, G. Linnell, and B. Ratkovits Discrepancy Rates of Radiology Resident Interpretations of On-Call Neuroradiology MR Imaging Studies Radiology, December 1, 2008; 249(3): 972 - 979. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 W.M. Strub, J.L. Leach, T. Tomsick, and A. Vagal Overnight Preliminary Head CT Interpretations Provided by Residents: Locations of Misidentified Intracranial Hemorrhage AJNR Am. J. Neuroradiol., October 1, 2007; 28(9): 1679 - 1682. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R. B. Ruchman, J. Jaeger, E. F. Wiggins III, S. Seinfeld, V. Thakral, S. Bolla, and S. Wallach Preliminary Radiology Resident Interpretations Versus Final Attending Radiologist Interpretations and the Impact on Patient Care in a Community Hospital Am. J. Roentgenol., September 1, 2007; 189(3): 523 - 526. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
