AJR Women's Imaging Online
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
 QUICK SEARCH:   [advanced]


     


This Article
Right arrow Abstract Freely available
Right arrow Figures Only
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Title, R. S.
Right arrow Articles by Tello, R.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Title, R. S.
Right arrow Articles by Tello, R.
Social Bookmarking
 Add to CiteULike   Add to Complore   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Reddit   Add to Technorati  
What's this?
Hotlight (NEW!)
Right arrow
What's Hotlight?
DOI:10.2214/AJR.04.1171
AJR 2005; 185:1240-1244
© American Roentgen Ray Society


Original Research

Observer Performance in Assessing Anemia on Thoracic CT

Rachel S. Title1,2, Keith Harper1, Erik Nelson1, Tom Evans1 and Richard Tello1,3

1 Department of Radiology, Boston Medical Center, Boston University, Boston, MA 02118.
2 Present address: Department of Radiology, New York-Presbyterian Hospital, New York Weill Cornell Medical Center, 525 E 68th St., Starr Pavilion, Box 141, New York, NY 10021.
3 Deceased.

Received July 25, 2004; revised December 6, 2004;

 
Address correspondence to R. S. Title.


Abstract
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
OBJECTIVE. The purpose of this study was to evaluate the ability of expert reviewers to differentiate an anemic from a nonanemic state on the basis of visual assessment of the relative attenuation of blood in the left ventricle on noncontrast thoracic CT images and to compare reviewer performance with quantitative measurement of CT density in Hounsfield units.

MATERIALS AND METHODS. One hundred two noncontrast thoracic CT examinations were qualitatively reviewed by three independent reviewers. Hounsfield unit measurements of the blood in the left ventricle were recorded by a fourth individual. Anemia was defined as a hemoglobin level of less than 10 g/dL. Receiver operating characteristic (ROC) analyses of expert reviewers were compared with measured Hounsfield units.

RESULTS. Hounsfield unit measurements performed significantly better than subjective reviewer analyses for differentiation of an anemic from a nonanemic state (area under ROC curve = 0.85 vs 0.72, 0.70, and 0.69; 95% confidence interval, 0.78–0.92 vs 0.63–0.81, 0.61–0.79, and 0.60–0.78, respectively; p < 0.05). With use of a CT density threshold of 35 H, the sensitivity for anemia was 76% and specificity was 81%, whereas the sensitivity of three reviewers was 40–72% with a specificity of 60–83%. Interobserver agreement was found to be poor by kappa statistic (0.0906–0.2128). The correlation coefficient for the analysis of Hounsfield unit versus hemoglobin level was 0.72. Separating data by patient sex revealed a correlation coefficient of 0.81 for men versus 0.52 for women, although the actual regression lines were not statistically different (p > 0.05).

CONCLUSION. Despite expert reviewer analyses, subjective evaluations of blood attenuation characteristics are prone to inaccuracy and show poor interobserver agreement. Quantitative measurements of CT density in Hounsfield units should be performed to accurately differentiate an anemic from a nonanemic state when serum hemoglobin levels are not readily available.


Introduction
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
Although anemia is not primarily diagnosed on CT and serum hemoglobin levels can be readily obtained, it is often that an anemic state may prompt CT. Experienced radiologists are able to visually appreciate minor variations in density between adjacent tissues on CT images. During interpretation of noncontrast CT scans of the chest or abdomen performed for other indications, some radiologists may comment on the density of the blood within the cardiac chambers or major vessels and may infer an anemic state. It is assumed by many radiologists that one can usually correctly identify anemia (hemoglobin, < 10 g/dL) in patients simply by comparing the attenuation of blood in noncontrast studies with the adjacent vessel wall or myocardium [1]. A study by Powell and colleagues [1] revealed that in the anemic state, the right and left ventricular cavities become clearly visible in canine hearts, thus making it possible to distinguish readily the boundaries of the ventricular and atrial cavities, the papillary muscles, the major trabeculae, and the aorta. Conventional wisdom and anecdotal clinical experience support these findings; however, their validity in the in vivo human heart has not to our knowledge been studied extensively.

Hounsfield units (H) are the units of measurement of density on CT. In CT, a number (between –1,000 and 1,000 H) is assigned by computer to represent the difference in X-ray attenuation between a given material and water (where air is –1,000 H and pure water is 0 H). This number is then used by the computer to assign a gray-scale shade to the represented image. The exquisite contrast resolution of current CT scanners can detect differences in contrast of less than 5 H (< 0.5%).



View larger version (5K):
[in this window]
[in a new window]
[as a PowerPoint slide]
 
Fig. 1 Graph shows correlation between density in Hounsfield units and hemoglobin levels in all 102 patients.

 



View larger version (4K):
[in this window]
[in a new window]
[as a PowerPoint slide]
 
Fig. 2 Graph shows correlation between hemoglobin levels versus density in Hounsfield units in men.

 
Density measurements can play a crucial role in the diagnosis and management of certain disease processes. CT has been shown to be useful in both the initial diagnosis of iron overload in the liver and as a method of following the course of treatment [2]. Density measurements of the liver have been found to correlate with serum ferritin levels in patients with ß-thalassemia major [3]. CT density measurements of intravascular blood by Di Giandomenico et al. [4] showed a significant difference in blood attenuation values between healthy subjects and anemic patients. This study identified borderline density values of 33 H for women and 36 H for men, below which the diagnosis of anemia can be made with 99% confidence.



View larger version (4K):
[in this window]
[in a new window]
[as a PowerPoint slide]
 
Fig. 3 Graph shows correlation between hemoglobin levels versus density in Hounsfield units in women.

 



View larger version (3K):
[in this window]
[in a new window]
[as a PowerPoint slide]
 
Fig. 4 Graph shows receiver operating characteristic curve analysis of CT density in Hounsfield units as discriminator of anemia with hemoglobin level of less than 12 as cutoff.

 



View larger version (3K):
[in this window]
[in a new window]
[as a PowerPoint slide]
 
Fig. 5 Graph shows receiver operating characteristic curve fit for binormal curves for ability of all observers to discriminate anemia from nonanemia.

 
Despite the presence of evidence of the efficacy of objective Hounsfield unit measurements in the assessment of anemia, we have observed that the diagnosis of anemia is rendered by reviewers on the basis of only a visual inspection and that objective measurement is seldom performed for confirmation, a practice that sometimes causes consternation among our clinical colleagues. There are, to our knowledge, no data regarding the accuracy of assessing anemia by visual inspection of blood attenuation on CT images. Furthermore, studies of the accuracy of objective measurements have also been scarce, and sample sizes of these studies have been small.

The aims of this study were to evaluate the ability and reliability of expert reviewers to differentiate an anemic from a nonanemic state on the basis of a visual assessment of the attenuation of blood in the left ventricle on noncontrast thoracic CT images and to compare this subjective evaluation with density measured in Hounsfield units. We attempted to determine if subjective (visual assessment) or objective (measurement of Hounsfield units within the lumen of the left ventricle) evaluation was sufficiently accurate in differentiating anemic from nonanemic states and whether quantitative measurements of Hounsfield units are required for confident diagnosis.


Materials and Methods
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
Criteria
Institutional review board approval was obtained for a retrospective review of all adult thoracic CT examinations acquired at our academic medical center between June 28, 2001, and August 31, 2001, to identify noncontrast studies. Of the 251 noncontrast thoracic CT examinations performed during that time period, only those studies of patients with a hemoglobin analysis within 24 hr of the time of the study and who did not undergo an acute change in hemoglobin level over the 3 days after CT were subjected to further review.

Statistical Design
Using the method of Hanley and McNeil [5] for the same population of subjects and a power of 0.8, at least 25 abnormal and 65 normal cases would be required to perform an adequate analysis at the 0.05 cutoff.

Demographics
A cohort of 102 patients was identified on the basis of these criteria. There were 61 men and 41 women who ranged in age from 21 to 93 years (mean, 56.8 years). The hemoglobin levels ranged from 7.0 to 18.2 g/dL (mean, 11.6 g/dL; normal laboratory values: men, > 13.5 g/dL, and women, >12 g/dL). Anemia was defined as a hemoglobin level of less than 10 g/dL. Twenty-five of the 102 patients had a hemoglobin level of less than 10 g/dL.

CT examinations were performed on two different CT scanners. Fifty-six patients (41 men, 15 women) underwent CT on an MDCT scanner (MX 8000, Philips Medical Systems) using 3.2-mm thickness, 1.25 pitch, 200 mAs, and 140 kVp; the remaining 46 patients (20 men, 26 women) underwent CT on a helical scanner (PQ 5000, Philips Medical Systems) using 5-mm thickness, 1.5 pitch, 225 mAs, and 120 kVp.

Qualitative Image Analysis
One hundred two noncontrast thoracic CT examinations were visually reviewed independently by three experienced thoracic CT reviewers blinded to each other's readings and to the anemic state of the patient using soft-tissue window settings (window, 400 H; level, 30 H) on a commercial PACS-based workstation (Amicas, Version 3.5.5, Amicas). Based on visual assessment of the perceived attenuation of blood in the left ventricle, observers were asked to classify the patient as definitely anemic, probably anemic, indeterminate, probably not anemic, or definitely not anemic using a 5-point scale, which is similar to other subjective reviewer scales validated previously [6]. The reviewers were shown two sample cases representing the patients with the lowest and highest hemoglobin levels to standardize reviewer interpretation.

Quantitative Image Analysis
The density in Hounsfield units of the blood in the left ventricle was measured by a fourth observer. An oval or circular region ranging in area from 3.0 to 3.9 cm2 (mean, 3.5 cm2) located within the confines of the lumen of the left ventricle was selected. The Hounsfield unit measurements were entered into a computer database.

Statistical Analysis
True-positive cases were defined as anemic patients correctly identified. True-negative cases were defined as nonanemic patients that were correctly identified. Interobserver agreement was calculated using the kappa statistic [7]. Receiver operating characteristic (ROC) curves were calculated to compare the results of the three reviewers with the density measurements in Hounsfield units [810]. The diagnostic accuracy was determined by calculating the area under each ROC curve (Az) [11]. The Az values for the three reviewers and for density measurements in Hounsfield units were compared using the method of proportions analysis. All statistical analyses were performed using STATA software (Stata).


Results
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
The mean density measured in Hounsfield units for anemic patients was 31.8 H with a range from 19.3 to 47.9 H and that for nonanemic patients, 42.8 H with a range from 15.1 to 61.7 H. Figure 1 is a graph representing the correlation of Hounsfield unit measurements to hemoglobin level. The correlation coefficient for the analysis of Hounsfield unit values compared with hemoglobin level was 0.72. Separating data by patient sex revealed a greater correlation of Hounsfield unit values to hemoglobin levels for men compared with that for women (correlation coefficient 0.81 for men vs 0.52 for women), which is shown in Figures 2 and 3, respectively, although the actual regression lines were not statistically different by Chow's test [12] (p > 0.05). Separating data by scanner type revealed only a modest nonsignificant difference between correlation coefficients (0.77 for the MDCT scanner vs 0.63 for the helical scanner).

Statistical analyses using STATA software for different hemoglobin levels established that the greatest area under the ROC curve (Az) was obtained for a hemoglobin level of 12 g/dL (Az = 0.87). Using this ROC curve, we determined the Hounsfield unit value closest to a sensitivity of 1.0 and specificity of 1.0. The ROC curve for density measured in Hounsfield units (Fig. 4) revealed that the use of a Hounsfield unit value of 35 H to define anemia produces the most accurate results in terms of maximizing true-positives and minimizing false-negatives. With the use of a Hounsfield unit threshold of 35 H, the sensitivity of Hounsfield unit measurements for the anemic state was 76% and specificity was 81%.

The sensitivity of the three reviewers for determining the anemic state ranged from 40% to 72% and specificity ranged from 60% to 83%. Interobserver variability was found to be very high by kappa analysis. The interobserver kappa values for the three reviewers were 0.0906, 0.0914, and 0.2128 (poor), where a kappa value of 1 represents complete agreement between reviewers. Seven of 25 anemic patients were correctly classified by all three reviewers as definitely or probably anemic, and 24 of 77 nonanemic patients were correctly classified by all three reviewers as definitely or probably not anemic.

ROC curves for measured Hounsfield units and for the three independent reviewers for distinguishing anemic from nonanemic states are shown in Figures 4 and 5. The mean area under the ROC curve (Az) for measured Hounsfield units was significantly higher than that for the individual reviewer analysis (0.85 vs 0.72, 0.70, and 0.69; 95% confidence intervals, 0.78–0.92 vs 0.63–0.81, 0.61–0.79, and 0.60–0.78 for reviewers 1, 2, and 3, respectively; p < 0.05).


Discussion
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
This study is, to our knowledge, the first to attempt to evaluate the validity of subjective visual assessment of density on CT images in the diagnosis of anemia. In addition, we attempted to reassess the validity of objective assessment of anemia with Hounsfield unit measurements as discussed in previous smaller studies [4]. The results of this study show that Hounsfield unit measurements performed significantly better than subjective reviewer analyses for differentiating an anemic from a nonanemic state. Subjective (visual) assessment of anemia on thoracic CT proved to be prone to inaccuracy and a high degree of interobserver variability. This method was found to be unreliable, and the data suggest that even if anemia is suspected by visual assessment, objective measurement of density values in Hounsfield unit of the ventricular blood is warranted for more confident diagnosis.

There are several potential limitations of this study. Patients were studied on two different types of CT scanners, and Hounsfield unit measurements are known to vary among scanners because of maladjustments, shortand long-term drift, X-ray fan, and detector asymmetry, which could lead to nonuniformity of data. However, a study looking at interscanner conformity in CT densitometry of the lungs concluded that after correction for poor air calibration, scanner conformity was acceptable [13], and correlation of data between the two scanners used in this study was high.

Another limitation to the study is that it is difficult to definitively distinguish intracardiac structures in nonanemic patients because of the lack of a differential between cavitary and myocardial densities [1]. Thus, one cannot be certain that the oval region within the confines of the lumen of the left ventricle selected for measurement of Hounsfield units did not contain any myocardium. Finally, a subjective difference exists in the way that CT images are visually interpreted in clinical practice and the necessary method of data collection for this study. The three observers in this study examined each CT scan to specifically analyze the images of the heart to evaluate for anemia and then rated the severity or certainty of the finding. In clinical practice, the examination is usually performed for an indication completely unrelated to anemia, and the question of anemia might arise only if one observes a striking contrast between luminal blood and adjacent myocardium. Therefore, a random observer's threshold for making the diagnosis of anemia may differ from that of the observers in this study, and more subtle cases might be ignored in clinical practice.

The fact that the correlation of Hounsfield unit values and hemoglobin levels proved greater in men than in women is a curious finding that may be attributable to breast attenuation or body fat distribution and merits further investigation. Although this was not the goal of our study, the different correlations are significant and because the same hemoglobin cutoff levels were used for men and women, in contrast to other studies [4], a confounding variable could be menstruation or other causes of active bleeding. At our site, unstable patients are unstable predominantly as a result of trauma, which indicates that these cases were automatically excluded because unstable patients are usually given IV contrast material.

In conclusion, despite expert reviewer analyses, subjective evaluations of blood attenuation characteristics are prone to inaccuracy and show high interobserver variability (i.e., lack certainty and consistency). If a diagnosis of anemia is questioned clinically or due to subjective visual analysis of the CT images, quantitative measurements of density in Hounsfield units should be performed to accurately differentiate an anemic from a nonanemic state when serum hemoglobin information may be lacking. In the measurement of the CT attenuation of blood in the lumen of the left ventricle, a threshold value of 35 H was found to best separate anemic from nonanemic states.


References
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 

  1. Powell WJ, Wittenberg J, Dinsmore RE, Miller SW, Maturi RA. Definition of cardiac structures using computerized tomography in isolated arrested and beating canine hearts. Am J Cardiol1977; 39:690 –696[CrossRef][Medline]
  2. Mills SR, Doppman JL, Nienhuis AW. Computed tomography in the diagnosis of disorders of excessive iron storage of the liver. J Comput Assist Tomogr 1977; 1:101 –104[Medline]
  3. Babiker MA, Patel PJ, Karrar ZA, Hafeez MH. Comparison between serum ferritin and computed tomographic densities of liver, spleen, kidney and pancreas in beta-thalassaemia major. Scand J Clin Lab Invest 1987; 47:715 –718[Medline]
  4. Di Giandomenico E, Genovesi N, Sciarra R, Angelone T, Toppetti A. Quantitative evaluation of blood in vivo with computed tomography [in Italian]. Radiol Med (Torino) 1993;85 : 416–420
  5. Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983;148 : 839–843[Abstract/Free Full Text]
  6. Fenlon HM, Tello R, deCarvalho VLS, Yucel EK. Signal characteristics of focal liver lesions on double echo T2-weighted conventional spin echo MRI: observer performance versus quantitative measurement of T2 relaxation times. J Comput Assist Tomogr2000; 24:204 –211[CrossRef][Medline]
  7. Fleiss JL. The measurement of interrater agreement: statistical methods for rates and proportions, 2nd ed. New York, NY: Wiley, 1981:212 –236
  8. Metz CE. ROC methodology in radiologic imaging. Invest Radiol 1986; 21:720 –733[Medline]
  9. Mann FA, Hildebolt CF, Wilson AJ. Statistical analysis with receiver operating characteristic curves. Radiology1992; 184:37 –38[Free Full Text]
  10. Vining DJ, Gladish GW. Receiver operating characteristic curves: a basic understanding. RadioGraphics 1992;12 :1147 –1154[Abstract]
  11. Brismar J. Understanding receiver-operating-characteristic curves: a graphic approach. AJR 1991;157 :1119 –1121[Free Full Text]
  12. Chow GC. Test of equality between sets of coefficients in two linear regressions. Econometrica 1960;28 : 591–605[CrossRef]
  13. Kemerink GJ, Lamers RJ, Thelissen GR, van Engelshoven JM. Scanner conformity in CT densitometry of the lungs. Radiology1995; 197:749 –752[Abstract/Free Full Text]

Add to CiteULike CiteULike   Add to Complore Complore   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Reddit Reddit   Add to Technorati Technorati    What's this?



This Article
Right arrow Abstract Freely available
Right arrow Figures Only
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Title, R. S.
Right arrow Articles by Tello, R.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Title, R. S.
Right arrow Articles by Tello, R.
Social Bookmarking
 Add to CiteULike   Add to Complore   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Reddit   Add to Technorati  
What's this?
Hotlight (NEW!)
Right arrow
What's Hotlight?


HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS