AJR Your Link to CME
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
 QUICK SEARCH:   [advanced]


     


This Article
Right arrow Abstract Freely available
Right arrow Figures Only
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Middleton, W. D.
Right arrow Articles by Yamaguchi, K.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Middleton, W. D.
Right arrow Articles by Yamaguchi, K.
Social Bookmarking
 Add to CiteULike   Add to Complore   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Reddit   Add to Technorati  
What's this?
AJR 2004; 183:1465-1468
© American Roentgen Ray Society

Sonography of the Rotator Cuff: Analysis of Interobserver Variability

William D. Middleton1, Sharlene A. Teefey1 and Ken Yamaguchi2

1 Mallinckrodt Institute of Radiology, Washington University School of Medicine, 510 S Kingshighway Blvd., St. Louis, MO 63110.
2 Department of Orthopedic Surgery, Washington University School of Medicine, St. Louis, MO 63110.

Received January 19, 2004; accepted after revision March 8, 2004.

 
Address correspondence to W. D. Middleton.


Abstract
Top
Abstract
Introduction
Subjects and Methods
Results
Discussion
References
 
OBJECTIVE. The purpose of our study was to determine the interobserver variability in the sonographic evaluation of the rotator cuff.

SUBJECTS AND METHODS. Two radiologists independently scanned 61 patients with shoulder pain. Each radiologist categorized the rotator cuff as normal, partially torn, or fully torn. When a tear was present, the tendons involved were specified. All diagnoses were made prospectively without knowledge of the findings of the other radiologist.

RESULTS. The radiologists were in full agreement in the categorization of 92% (56/61) of the patients. In four of the five discrepant cases, the disagreement was whether there was a full-thickness or a partial-thickness tear. The radiologists were in agreement concerning which tendons were involved in 80% (41/51) of the patients in whom a tear was detected by both observers. In all 10 discrepant cases, the disagreement was whether a tear involved both the supraspinatus and infraspinatus tendons or was isolated to one or the other of these tendons.

CONCLUSION. The level of interobserver variability in the sonographic detection and characterization of rotator cuff tears is low.


Introduction
Top
Abstract
Introduction
Subjects and Methods
Results
Discussion
References
 
In patients with shoulder pain, one of the most important initial determinations is whether there is a rotator cuff tear [1]. Currently, two primary imaging methods are used to detect a torn rotator cuff. MRI is an excellent means of detecting full-thickness rotator cuff tears and has a proven interobserver variability that is acceptable [24]. Sonography has also been used for many years to diagnose rotator cuff tears [57]. Studies during the last decade have shown that sonography of the shoulder is as effective as MRI in revealing these tears [810]. However, sonography depends greatly on the skill of the operator, and there is little documentation that the examination is reproducible. The purpose of our study was to test the interobserver variability in the detection of rotator cuff tears on shoulder sonography.


Subjects and Methods
Top
Abstract
Introduction
Subjects and Methods
Results
Discussion
References
 
Between December 1998 and April 2001, 130 patients with a high clinical suspicion for a rotator cuff tear were referred for imaging and considered potential candidates for the study. All of these patients had been initially evaluated clinically by a single orthopedic surgeon who is subspecialty-trained in shoulder and elbow surgery. Of these 130 patients, 61 patients were scanned by two completely independent observers who were both board-certified radiologists. Each radiologist had more than 5 years of experience with shoulder sonography when the study started. Each observer was blinded to the images and the interpetation of the other observer. The observer who performed the initial scanning was randomized. Because of scheduling conflicts with one or the other of the observers, the remaining 69 patients were only scanned once and were excluded from the study. There was no preselection of patients who would be scanned twice other than the availability of the second observer. The human studies committee of our institution approved the study, and written informed consent was obtained from all patients before their examination. Our study population was 35 men and 26 women whose ages ranged from 18 to 80 years, with a mean age of 58.2 years.

Both observers used similar scanning protocols. Patients were scanned while seated. The subscapularis tendon was examined in long and short axis with the shoulder externally rotated. The supraspinatus and infraspinatus tendons were scanned in long and short axis with the hand positioned on the posterior hip and the elbow flexed and directed posteriorly. All examinations were performed with a Siemens Elegra unit using a 7.5-MHz linear array transducer. Imaging parameters such as scanning frequency, focal zone number and placement, gain, field of view, and fundamental versus harmonic imaging mode were not standardized but left to the discretion of the observer. Most patients were scanned in the harmonic mode with a transmit frequency of 4.5 MHz.

After the examinations, each observer filled out a data sheet. The rotator cuff was graded as normal or torn. Torn cuffs were further graded as full-thickness, partial-thickness, or indeterminate tear (i.e., the observer can see that the cuff is torn but is unable to distinguish between a full- or partial-thickness tear). In addition, tears were localized to the subscapularis, supraspinatus, infraspinatus, or a combined tear of more than one tendon. The supraspinatus tendon was defined as the portion of the superior cuff within 1.5 cm of the biceps tendon. The remainder of the posterior cuff that was more than 1.5 cm posterior to the biceps tendon was considered the infraspinatus tendon. For the purpose of this study, the teres minor and the infraspinatus tendon were considered a single unit.


Results
Top
Abstract
Introduction
Subjects and Methods
Results
Discussion
References
 
The two observers disagreed about the status of the rotator cuff in five of the 61 patients scanned. In the cases of three patients, one of the two observers considered the cuff torn but indeterminate, while the other observer considered the cuff partially (n = 2) or fully torn (n = 1). These were the only three patients for whom the indeterminate tear category was used. Surgery was not performed on these three patients. In a fourth patient, one observer graded the cuff as fully torn and the other observer graded the cuff as partially torn. This patient was found at surgery to have a partial-thickness tear. In the fifth patient, one observer graded the cuff as normal and the other observer graded the cuff as partially torn (Fig. 1A, 1B). This patient was found to have a partial-thickness tear at surgery.



View larger version (93K):
[in this window]
[in a new window]
[as a PowerPoint slide]
 
Fig. 1A. 47-year-old man with left shoulder pain. Arthroscopy performed after sonographic examination showed partial-thickness tear of supraspinatus tendon. Long-axis sonographic view of left supraspinatus insertion obtained from observer 1 shows hypoechoic region (cursors) that was interpreted as deep surface partial-thickness tear.

 


View larger version (100K):
[in this window]
[in a new window]
[as a PowerPoint slide]
 
Fig. 1B. 47-year-old man with left shoulder pain. Arthroscopy performed after sonographic examination showed partial-thickness tear of supraspinatus tendon. Similar long-axis sonographic view of left supraspinatus insertion obtained from observer 2 shows hypoechoic region similar to that shown in A that was interpreted as normal anisotropy.

 

The two observers were in full agreement about the status of the rotator cuff in the remaining 56 patients (92%). The sonograms of 38 of these patients were interpreted as showing full-thickness tears (Fig. 2A, 2B); nine, as showing partial-thickness tears (Fig. 3A, 3B); and nine, as showing normal rotator cuffs. Thirty-five of these 56 patients underwent surgery. All 23 full-thickness tears detected surgically were correctly classified sonographically. All eight partial-thickness tears detected surgically were detected sonographically; however, four were classified as full-thickness tears and four were classified as partial-thickness tears. Of the four rotator cuffs found to be normal at surgery, three were classified as normal on sonography and one was classified as a partial-thickness tear.



View larger version (87K):
[in this window]
[in a new window]
[as a PowerPoint slide]
 
Fig. 2A. 75-year-old man with left shoulder pain. Surgery was not performed on this patient. Short-axis sonographic view of left supraspinatus and infraspinatus insertions obtained from observer 1 shows rotator cuff defect (cursors) that was interpreted as full-thickness tear of supraspinatus and infraspinatus tendons.

 


View larger version (94K):
[in this window]
[in a new window]
[as a PowerPoint slide]
 
Fig. 2B. 75-year-old man with left shoulder pain. Surgery was not performed on this patient. Similar short-axis sonographic view of left supraspinatus and infraspinatus insertions obtained from observer 2 shows rotator cuff defect (cursors) that was also interpreted as a full-thickness tear of supraspinatus and infraspinatus tendons.

 


View larger version (99K):
[in this window]
[in a new window]
[as a PowerPoint slide]
 
Fig. 3A. 65-year-old woman with left shoulder pain. Arthroscopy performed after sonographic examination showed partial-thickness tear of supraspinatus tendon. Long-axis view of left supraspinatus insertion obtained from observer 1 shows hypoechoic defect (cursors) that was interpreted as partial-thickness tear.

 


View larger version (100K):
[in this window]
[in a new window]
[as a PowerPoint slide]
 
Fig. 3B. 65-year-old woman with left shoulder pain. Arthroscopy performed after sonographic examination showed partial-thickness tear of supraspinatus tendon. Similar long-axis sonographic view of left supraspinatus insertion obtained from observer 2 shows similar hypoechoic defect (cursors) that was also interpreted as partial-thickness tear.

 

Both observers agreed about the location of the tear and the tendons involved in 41 of the 51 patients who had a tear (either partial- or full-thickness) that was detected by both observers. In 27, the tear involved both the supraspinatus and the infraspinatus tendons. In 13, the tear was isolated to the supraspinatus tendon. In one patient, the tear was isolated to the infraspinatus tendon. In 10 patients, the observers disagreed concerning which tendons were involved. In nine patients, one of the observers thought that the tear was isolated to the supraspinatus tendon, whereas the other observer thought that the tear involved both the supraspinatus and the infraspinatus tendons. In one patient, one observer thought that the tear was isolated to the infraspinatus tendon, whereas the other observer thought that the tear involved both the supraspinatus and the infraspinatus tendons.


Discussion
Top
Abstract
Introduction
Subjects and Methods
Results
Discussion
References
 
Many studies have documented the accuracy of sonography in the detection of full-thickness rotator cuff tears [810]. Sensitivities range from 94% to 100%; specificities range from 91% to 94%. Sonography has several advantages over MRI that make it an attractive alternative. It is considerably faster, it can be used on claustrophobic patients, it can be used for bilateral examinations, and it is considerably less expensive. Accordingly, sonography is being used with increased frequency as the initial means of evaluating patients with a painful shoulder.

On the other hand, MRI is equally accurate [1114], it gives a more global evaluation of the shoulder, and it has a shorter learning curve; consequently many radiologists are trained to interpret shoulder MR images. Perhaps as important as any of the advantages mentioned above, MRI has been shown in several studies to be a reproducible technique with low interobserver variability for detection of full-thickness tears [3, 4].

Because sonography is rightfully viewed as one of the most operator-dependent imaging examinations, and examination of the shoulder is considered one of the more challenging aspects of sonography, one might expect that there would be high interobserver variability for this examination. However, to our knowledge, there has been no systematic study specifically analyzing the interobserver variability for the examination. Undoubtedly, one of the impediments of investigating interobserver variability in shoulder sonography is the intrinsic difference in the way in which sonographic images are acquired compared with the way in which images are acquired with other techniques, such as MRI. With sonography, the recorded images largely display the findings observed by the individual performing the examination. If the examiner detects a rotator cuff tear, the recorded images show a tear. If the examiner misses a rotator cuff tear, then the recorded images do not show a tear. Because of these factors, it is not realistic to measure interobserver variability by having different observers interpret the same set of images. It actually requires having two independent examiners perform two completely separate examinations and interpret the results on the basis of real-time examinations and their own set of stored images.

The latter approach was adopted for this study, and the results confirm that interobserver variability is low. The two observers agreed on the classification of the cuff status in 92% of patients. The observers were in disagreement for only five patients. Three disagreements occurred when both observers detected a tear, but one of the observers thought that it was not possible to confidently categorize the tear as a full- or partial-thickness tear and therefore placed it in the indeterminate tear category. These were the only three patients with tears for which the indeterminate tear category was used. In the other two discrepant cases, the difference in categorization was partial-thickness tear versus normal and partial-thickness tear versus full-thickness tear. Thus, there was only a single case in which the difference in categorization was between a normal versus torn rotator cuff, and there were no cases in which one observer diagnosed a full-thickness tear and the other diagnosed a normal rotator cuff. Therefore, even when there was a disagreement, the discrepancy was typically minor.

As part of this study, the observers were also required to determine the rotator cuff tendon or tendons that were involved in the tear. In 80% (41/51) of the patients in whom both observers detected a tear, the observers agreed on the tendons involved. In the remaining 10 patients, the disagreements all occurred when one of the two observers thought that the tear involved both the supraspinatus and the infraspinatus tendons, whereas the other observer thought that the tear was isolated to either the supraspinatus (n = 9) or the infraspinatus (n = 1) tendon. We believe that these disagreements were relatively minor and are unlikely to have a significant impact in the treatment of these patients.

There are limitations to our study. The mix of patients reflects the type of population seen by a subspecialized shoulder surgeon who primarily sees referred patients who have a high likelihood of having significant structural abnormalities. For this reason, most patients enrolled had a torn rotator cuff, and only 15% of patients had normal cuffs. This high percentage of patients with torn cuffs is unlikely to match most patient populations referred for shoulder sonography. However, if anything, we believe that the high percentage of torn cuffs is more likely to make it more difficult for observers to agree. As our study showed, it is quite uncommon for a normal cuff to be confused with a torn cuff. However, distinguishing an extensive partial-thickness tear from a nonretracted full-thickness tear can be problematic [8]. In fact, this situation accounted for all but one of the discrepancies in our study. If there were more normal rotator cuffs and fewer torn cuffs, we believe that our rate of agreement would have improved.

A second limitation is the lack of surgical proof in all patients. Although such proof would have been desirable, many factors influence the choice of whether to perform surgery in patients with a rotator cuff tear, and some patients or their physicians opted for conservative treatment. As mentioned previously, many studies have documented the sensitivity of sonography for the detection of rotator cuff tears. In a recent technology assessment investigation that included analysis of 38 cohort studies for sonography and 29 for MRI, Dinnes et al. [14] found that either test could be used equally for detection of full-thickness rotator cuff tears. Accordingly, the focus of this study was to evaluate interobserver variability, not to reconfirm the accuracy. We believe that we were justified to include the patients who lacked surgical confirmation.

The final limitation relates to the observers. Both were experienced operators, each having more than 5 years of experience scanning shoulders. If a less experienced operator had participated in the study, it is likely that the variability would have been greater [13]. At our institution, only two radiologists perform shoulder sonography; a third less experienced operator was not an option. This limitation emphasizes one of the major problems with shoulder sonography. Only a limited number of individuals have the expertise to scan shoulders, and this fact has retarded growth of the technique. Hopefully, our results dealing with the reproducibility of the examination will add to the existing body of literature that shows that sonography is an accurate method and will stimulate more radiologists to become trained in shoulder sonography.

In summary, we have shown that in experienced hands, sonography has a low level of interobserver variability for the detection, classification, and localization of rotator cuff tears.


References
Top
Abstract
Introduction
Subjects and Methods
Results
Discussion
References
 

  1. Glockner SM. Shoulder pain: a diagnostic dilemma. Am Fam Physician 1995;51:1677 –1687[Medline]
  2. Robertson PL, Schweitzer ME, Mitchell DG, et al. Rotator cuff disorders: interobserver and intraobserver variation in diagnosis with MR imaging. Radiology1995; 194:831 –835[Abstract/Free Full Text]
  3. Balich SM, Sheley RC, Brown TR, Sauser DD, Quinn SF. MR Imaging of the rotator cuff tendon: interobserver agreement and analysis of interpretive errors. Radiology1997; 204:191 –194[Abstract/Free Full Text]
  4. Sahin-Akyar G, Miller TT, Staron RB, McCarthy DM, Feldman F. Gradient-echo versus fat-suppressed fast spin-echo MR imaging of rotator cuff tears. AJR1998; 171:223 –227[Abstract/Free Full Text]
  5. Middleton WD, Reinus WR, Totty WG, et al. Ultrasonographic evaluation of the rotator cuff and biceps tendon. J Bone Joint Surg Am 1986;68:440 –450[Abstract/Free Full Text]
  6. Mack LA, Matsen FA, Kilcoyne JF, et al. US evaluation of the rotator cuff. Radiology1985; 157:205 –209[Abstract/Free Full Text]
  7. Hodler J, Fretz CJ, Terrier F, et al. Rotator cuff tears: correlation of sonographic and surgical findings. Radiology1988; 169:791 –794[Abstract/Free Full Text]
  8. Teefey SA, Hasan SA, Middleton WD, Patel M, Wright RW, Yamaguchi K. Ultrasonography of the rotator cuff: a comparison of ultrasonographic and arthroscopic findings in one hundred consecutive cases. J Bone Joint Surg Am 2000;82:498 –504[Abstract/Free Full Text]
  9. Wiener SN, Seitz WH Jr. Sonography of the shoulder in patients with tears of the rotator cuff: accuracy and value for selecting surgical options. AJR 1993;160:103 –107[Abstract/Free Full Text]
  10. van Holsbeeck MT, Kolowich PA, Eyler WR, et al. US depiction of partial-thickness tear of the rotator cuff. Radiology1995; 197:443 –446[Abstract/Free Full Text]
  11. Bryant L, Shnier R, Bryant C, Murrell GAC. A comparison of clinical estimation, ultrasonography, magnetic resonance imaging, and arthroscopy in determining the size of rotator cuff tears. J Shoulder Elbow Surg 2002;11:219 –224[Medline]
  12. Teefey SA, Middleton WD, Rubin DA, Mirowitz SA, Hildebolt CF, Yamaguchi K. Detection of partial and full thickness rotator cuff tears in patients with a painful shoulder: a comparison of ultrasound, MRI, and arthroscopic surgery. Radiology 2000;217 [suppl]: 430[Abstract/Free Full Text]
  13. Chang CY, Wang SF, Chiou HJ, Ma HL, Sun YC, Wu HD. Comparison of shoulder ultrasound and MR imaging in diagnosing full-thickness rotator cuff tears. Clin Imaging2002; 26:50 –54[Medline]
  14. Dinnes J, Loveman E, McIntyre L, Waugh N. The effectiveness of diagnostic tests for the assessment of shoulder pain due to soft tissue disorders: a systematic review. Health Technol Assess2003; 7:1 –166

Add to CiteULike CiteULike   Add to Complore Complore   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Reddit Reddit   Add to Technorati Technorati    What's this?


This article has been cited by other articles:


Home page
J Bone Joint Surg BrHome page
J. L. Rees
The pathogenesis and surgical treatment of tears of the rotator cuff
J Bone Joint Surg Br, July 1, 2008; 90-B(7): 827 - 832.
[Abstract] [Full Text] [PDF]


Home page
J Bone Joint Surg BrHome page
A. Al-Shawi, R. Badge, and T. Bunker
The detection of full thickness rotator cuff tears using ultrasound
J Bone Joint Surg Br, July 1, 2008; 90-B(7): 889 - 892.
[Abstract] [Full Text] [PDF]


Home page
Ann Rheum DisHome page
J M Koski, S Saarakkala, M Helle, U Hakulinen, J O Heikkinen, H Hermunen, P Balint, G A Bruyn, E Filippucci, W Grassi, et al.
Assessing the intra- and inter-reader reliability of dynamic ultrasound images in power Doppler ultrasonography
Ann Rheum Dis, December 1, 2006; 65(12): 1658 - 1660.
[Abstract] [Full Text] [PDF]


Home page
Ann Rheum DisHome page
E Naredo, I Moller, C Moragues, J J de Agustin, A K Scheel, W Grassi, E de Miguel, M Backhaus, P Balint, G A W Bruyn, et al.
Interobserver reliability in musculoskeletal ultrasonography: results from a "Teach the Teachers" rheumatologist course
Ann Rheum Dis, January 1, 2006; 65(1): 14 - 19.
[Abstract] [Full Text] [PDF]


Home page
Am. J. Roentgenol.Home page
S. A. Teefey, W. D. Middleton, W. T. Payne, and K. Yamaguchi
Detection and Measurement of Rotator Cuff Tears with Sonography: Analysis of Diagnostic Errors
Am. J. Roentgenol., June 1, 2005; 184(6): 1768 - 1773.
[Abstract] [Full Text] [PDF]


This Article
Right arrow Abstract Freely available
Right arrow Figures Only
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Middleton, W. D.
Right arrow Articles by Yamaguchi, K.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Middleton, W. D.
Right arrow Articles by Yamaguchi, K.
Social Bookmarking
 Add to CiteULike   Add to Complore   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Reddit   Add to Technorati  
What's this?


HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS