HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Ashman, C. J. Articles by Wolfman, D. Search for Related Content PubMed PubMed Citation Articles by Ashman, C. J. Articles by Wolfman, D. Social Bookmarking                      What's this?

Satisfaction of Search in Osteoradiology

¹ All authors: Department of Radiology, The Ohio State University Medical Center, S-255 Rhodes Hall, 450 W. Tenth Ave., Columbus, OH 43210.

Received September 1, 1999; accepted after revision January 6, 2000.

 
Presented at the annual meeting of the American Roentgen Ray Society, New Orleans, May 1999.

Address correspondence to C. J. Ashman.

Abstract

Top Abstract Introduction Subjects and Methods Results Discussion References

 
OBJECTIVE. "Satisfaction of search" describes a situation in which the detection of one radiographic abnormality interferes with that of others. This phenomenon has not been investigated for the interpretation of skeletal radiographs. We determined whether satisfaction of search occurs in the interpretation of conventional radiographs of the musculoskeletal system.

SUBJECTS AND METHODS. Twelve volunteers from radiology (10 residents and two staff members) and 13 volunteers from orthopedic surgery (all residents) were shown in random order 15 cases in which one abnormality was present and 15 cases in which either two or three abnormalities were present. The cases with multiple findings contained a total of 19 abnormalities. All cases were selected from the emergency department. The volunteers received three scores reflecting their ability to detect abnormalities on a single-finding case, at least one abnormality on a multiple-finding case, and additional findings in cases with multiple abnormalities. The scores were compared using the chi-square test with Yates correction. Analysis of variance and Bonferroni adjustment were also performed.

RESULTS. Among the cases with single findings, detection rates averaged 11.25; for the 15 multiple-finding cases, the detection rate for one finding averaged 11.72. The difference was not statistically significant. Detection rates for the second and third abnormalities in the multiple-finding cases were significantly lower, averaging 6.12 (p < 0.01).

CONCLUSION. A significant satisfaction of search effect occurred and was operative in the interpretation of radiographs of the musculoskeletal system.

Introduction

Top Abstract Introduction Subjects and Methods Results Discussion References

 
Many radiologists find interpreting skeletal radiographs difficult. In part, this is likely related to the large number of normal anatomic variations in the musculoskeletal system, a broad range of disease processes that affect the skeletal system, and a lack of familiarity with numerous mechanisms of injury that often generate their own spectrum of associated injuries. Errors in diagnosis are problematic and are often in the form of a missed diagnosis (when a radiographic abnormality is not prospectively detected by an interpreter). Our radiology department interprets all studies performed in the emergency department. We have noticed a clear difference in the accuracy of interpretation, both by clinicians and residents, of radiographs with one abnormality when compared with those with more than one abnormality.

"Satisfaction of search" is a phrase that describes the phenomenon in which the detection of one radiographic finding interferes with that of others [1,2,3]. It has been found to apply to the interpretation of chest and abdominal radiographs, but it has not been determined whether it occurs in the interpretation of skeletal radiographs. Satisfaction of search has been used as an explanation for false-negative findings or underinterpreting (a source of diminished accuracy) in chest and abdominal radiography. We determined whether satisfaction of search occurs in the interpretation of skeletal radiographs and if it represented an important source of diminished accuracy in the examination of radiographs of the musculoskeletal system.

Subjects and Methods

Top Abstract Introduction Subjects and Methods Results Discussion References

 
Thirty radiographic examinations of the skeletal system were selected from patients examined in our emergency department for whom the clinical history was pain. No radiographs with normal findings were selected. Fifteen cases contained one significant diagnosis and 15 cases contained either two or three significant diagnoses. In the latter group, 11 cases contained two significant findings and four cases contained three findings. A significant diagnosis was considered one that had a major impact on the treatment plan or well-being of the patient. The verification of the abnormalities on the radiographs was made by subsequent imaging examinations, surgical reports, biopsy results, and the patients' clinical courses. The cases were arranged in random order on radiographic viewboxes.

The volunteers included 12 radiologists, comprising 10 residents and two attending radiologists, and 13 orthopedic surgery residents. A 75-min interpretation session was used for each group. The volunteers were unaware of the purpose of the study. Before starting the interpretation session, the film interpreters were given verbal and written instructions. They were informed there were a total of 30 cases, that no cases with normal findings were included, and that each case had one or more significant findings. Significant findings were described as those that would have a significant impact on the treatment or well-being of the patient; for example, a fracture or a metastasis was considered a significant finding. Finally, the volunteers were informed that most cases were obtained in the emergency department from patients presenting with pain. Each volunteer was given 2 min per case but was not prevented from exceeding this time. The volunteers were asked to record their answers on a standard form that contained a uniform-sized space to record their responses for each case.

Each participant's answer form was scored by one of the authors. Participants were scored for single-finding cases, for the detection of at least one finding on the multiple-finding cases, and for the detection of additional findings on the multiple-finding cases. The mean of each of these three scores was determined. The first and second and the second and third scores were compared with the chi-square test with Yates correction. Analysis of variance of the three scores was performed, followed by pairwise Bonferroni adjustment of the first and second, second and third, and first and third scores. The false-positive responses were scored for each volunteer for single-finding and multiple-finding cases, and the mean false-positive score was calculated for each set of cases. We defined a false-positive response as one that could not be attributed to the diagnosis in the single-finding case or to one of the diagnoses in the cases with multiple findings.

Results

Top Abstract Introduction Subjects and Methods Results Discussion References

 
The mean score for the cases with a single diagnosis, designated as score 1, was 11.25 of 15 (range, 7-14), with a standard deviation of 3.35. The mean score for the detection of at least one diagnosis on cases with multiple diagnoses, designated as score 2, was 11.72 of 15 (range, 10-14), with a standard deviation of 3.42. The difference between scores 1 and 2 was not statistically significant. The mean score for the detection of additional abnormalities in cases with multiple diagnoses, designated as score 3, was much lower than scores 1 and 2, at 6.12 of 19 (range, 2-11), with a standard deviation of 2.49. The chi-square test with Yates correction showed this difference to be statistically significant, as did analysis of variance with Bonferroni adjustment (p < 0.01).

The average number of false-positive responses was virtually identical among single- and multiple-finding cases, with a value for single-finding cases of 2.8 (range, 0-9) and for multiple-finding cases of 2.24 (range, 0-8).

The number of volunteers producing a satisfaction of search response for each multiple-finding case was determined (Tables 1,2,3). A satisfaction of search response was considered one in which at least one but fewer than the total number of correct diagnoses for a particular case was recorded. Cases with three findings elicited a greater satisfaction of search response than cases with two findings. Sixty percent or more of the volunteers showed a satisfaction of search response for all cases with three findings. Fifty-six percent or more of the volunteers showed a satisfaction of search response for five of the 11 cases with two findings. In one case, which depicted an acromioclavicular separation and rib fractures in a 36-year-old man, the seven satisfaction of search responses were nearly evenly split among the number of subjects who identified only finding A, the acromioclavicular separation, and those who recognized only finding B, the rib fractures. In the other cases, the satisfaction of search responses were unevenly divided with respect to the detection of finding A or B.

Discussion

Top Abstract Introduction Subjects and Methods Results Discussion References

 
Several investigators have shown that the satisfaction of search phenomenon occurs in the interpretation of chest radiographs [1,2,3,4,5]; however, the cause of the phenomenon is unknown. Tuddenham and Calvert [4] and Tuddenham [5] reported that the detection of one abnormality satisfied the "search for meaning" and resulted in a premature termination of the search. Berbaum et al. [1] studied the detection rate of single abnormalities on chest radiographs with and without the addition of simulated "distractor" pulmonary nodules. These investigators found a significant satisfaction of search effect with a decrease in the detection rates of native abnormalities in the presence of a distractor. Although the true-positive rate decreased in the presence of a distractor, the false-positive rate remained unchanged. However, the false-positive rate did increase with the true-positive rate in both situations. The postulated explanations for the satisfaction of search effect were a strategic termination of search to minimize the number of false-positive diagnoses and an adherence to a perceptual set, the inclination to discount findings of a different diagnostic category from the those found first [1]. In a subsequent study, Berbaum et al. [2] repeated the experiment but measured the inspection time for true-positive responses, false-positive responses, and the time between the last response and the termination of the search. A satisfaction of search effect was again found, but the search time was relatively constant and extended beyond the point at which abnormalities were reported, which did not support Tuddenham's quest for meaning theory. Also, although the true-positive rate did not increase with the time of search, the false-positive rate did, and the observers tended to terminate the search at a time before they were as likely or more likely to give a false-positive response than a true-positive response, further supporting the strategic termination of search hypothesis [2]. Franken et al. [3] reported a satisfaction of search effect in the detection of radiographic findings on abdominal contrast-enhanced images; however, their results differed in one respect from those reported by Berbaum et al. [1, 2]. Franken et al. found that the false-positive rate decreased with the true-positive rate and hypothesized that this decrease resulted from a lack of attention to regions that did not contain contrast material on contrast-enhanced images or a reluctance to report abnormalities on radiographs, instead of a failure to recognize abnormalities as previously described.

We detected a strong satisfaction of search effect in our experiment. Each abnormality among cases with multiple diagnoses was correctly identified by at least one radiographic interpreter (Tables 1 and 2). This indicated that the satisfaction of search effect was not a manifestation of the abnormalities being too subtle or too difficult to recognize. One case that showed bilateral rectus femoris avulsion fractures in a 12-year-old boy elicited the lowest satisfaction of search response. This was probably related to an unfamiliarity with the diagnosis because 22 interpreters overlooked both abnormalities.

This study was difficult to perform, and there were a number of limitations. Our experiment differed from previous satisfaction of search studies in that we could not provide a control group for cases with multiple findings because we had no means to remove one of the findings, a method used in previous investigations. It did not seem feasible or realistic to perform such a manipulation on radiographs of the skeletal system. Instead, we used cases with single findings, which were similar to the cases with multiple diagnoses in both difficulty and range of disorders; these single-finding radiographs served as our controls. This approach was validated by the fact that the first score, the mean detection rate of single-finding cases, and the second score, the detection rate of one finding on multiple-findings cases, were nearly identical. The cases with multiple findings were carefully selected so that each finding was considered significant and was similar to single-finding cases in terms of difficulty and range of disorders. Furthermore, we were careful to select radiographs that had identifiable findings that were not obscured by poor positioning, suboptimal exposure, or other technical shortcomings.

No radiographs with normal findings were used in our study. We hypothesized different potential effects of this approach. First, it may have controlled the false-negative rate by creating a situation that would not allow a participant to forego rendering a diagnosis if he or she was uncertain whether the finding could have been normal. In this way, the tendency to underinterpret would be controlled. If attempting to curb the false-negative rate increased the false-positive rate, this effect should have occurred equally in single-finding and multiple-finding cases. Alternatively, the absence of radiographs with normal findings could have decreased the false-positive rate by eliminating the contribution of overcalls on radiographs with normal findings to the false-positive rate. This would control a tendency to overinterpret. Because the false-positive rate was low in our study, the absence of radiographs with normal findings probably did not influence the overall satisfaction of search effect.

We don't know what effect experience has on the satisfaction of search phenomenon. For some of our volunteers, the inability to identify additional findings in the cases with multiple abnormalities may have been caused by a lack of experience. Knowledge of commonly associated abnormalities would be expected to be greater in individuals with more experience and may lessen the satisfaction of search effect. In this study, upper level residents and attending radiologists showed a satisfaction of search effect that was only slightly less than that of the lower level residents; however, the small number of volunteers in our study makes it difficult to speculate on whether the effect was significant. This is a potential area for further study.

Our objective was limited to assessing the presence of the satisfaction of search effect in the interpretation of skeletal radiographs; we did not investigate its cause. We did not measure search times and response times, so we could not determine whether finding an abnormality triggered a termination of search, as had been postulated in the quest for meaning hypothesis. The theory of strategic termination of search to limit the number of false-positive responses remains a possibility because the false-positive rate in our study remained relatively constant among single- and multiple-finding cases. The perceptual set theory does not seem to be a plausible explanation for our satisfaction of search effect because all but one of the strongly positive satisfaction of search cases contained abnormalities of the same pathologic process (trauma, neoplasm).

In summary, our experiment revealed that the satisfaction of search phenomenon takes place in the interpretation of skeletal radiographs and is a cause of inaccuracy in cases with multiple findings.

References

Top Abstract Introduction Subjects and Methods Results Discussion References

 

1. Berbaum KS, Franken EA Jr, Dorfman DD, et al. Satisfaction of search in diagnostic radiology. Invest Radiol 1990;25:133 -140[Medline]
2. Berbaum KS, Franken EA Jr, Dorfman DD, et al. Time course of satisfaction of search. Invest Radiol 1991;26:640 -648[Medline]
3. Franken EA Jr, Berbaum KS, Lu CH, et al. Satisfaction of search in the detection of plain-film abnormalities in abdominal contrast studies. Invest Radiol 1994;29:403 -409[Medline]
4. Tuddenham WJ, Calvert WP. Visual search patterns in roentgen diagnosis. Radiology 1961;76:255 -256[Medline]
5. Tuddenham WJ. Visual search, image organization, and reader error in roentgen diagnosis. Radiology 1962;78:694 -704

CiteULike

Complore

Connotea

Del.icio.us

Digg

Reddit

Technorati

This article has been cited by other articles:

				P.M. Phal, L.P. Riccelli, P. Wang, G.M. Nesbit, and J.C. Anderson Fracture Detection in the Cervical Spine with Multidetector CT: 1-mm versus 3-mm Axial Images AJNR Am. J. Neuroradiol., September 1, 2008; 29(8): 1446 - 1449. [Abstract] [Full Text] [PDF]

				R. A. W. Verhagen, M. Maas, M. G. W. Dijkgraaf, J. L. Tol, R. Krips, and C. N. van Dijk Prospective study on diagnostic strategies in osteochondral lesions of the talus: IS MRI SUPERIOR TO HELICAL CT? J Bone Joint Surg Br, January 1, 2005; 87-B(1): 41 - 46. [Abstract] [Full Text] [PDF]

				B. E. Van Beers, S. Dechambre, P. Hulcelle, R. Materne, and J. Jamart Value of Multislice Helical CT Scans and Maximum-Intensity-Projection Images to Improve Detection of Ureteral Stones at Abdominal Radiography Am. J. Roentgenol., November 1, 2001; 177(5): 1117 - 1121. [Abstract] [Full Text] [PDF]

				K. S. Berbaum, C. J. Ashman, J. S. Yu, and D. Wolfman Satisfaction of Search in Osteoradiology Am. J. Roentgenol., July 1, 2001; 177 (1): 252 - 253. [Full Text] [PDF]

				L. F. Rogers Keep Looking: Satisfaction of Search Am. J. Roentgenol., August 1, 2000; 175(2): 287 - 287. [Full Text] [PDF]

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Ashman, C. J. Articles by Wolfman, D. Search for Related Content PubMed PubMed Citation Articles by Ashman, C. J. Articles by Wolfman, D. Social Bookmarking                      What's this?