HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Figures Only Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Marmery, H. Articles by Mirvis, S. E. Search for Related Content PubMed PubMed Citation Articles by Marmery, H. Articles by Mirvis, S. E. Social Bookmarking                      What's this? Hotlight (NEW!) What's Hotlight?

Optimization of Selection for Nonoperative Management of Blunt Splenic Injury: Comparison of MDCT Grading Systems

¹ Department of Diagnostic Radiology, University of Maryland School of Medicine, 22 S Greene St., Baltimore, MD 21201.
² Present address: Department of Radiology, Nuffield Orthopaedic Hospital, Oxford, UK.
³ National Study Center for Trauma and Emergency Medical Systems, Baltimore, MD.

Received February 28, 2007; accepted after revision June 24, 2007.

 
Address correspondence to K. Shanmuganathan (kshanmuganathan{at}umm.edu).

Abstract

Top Abstract Introduction Materials and Methods Results Discussion References

 
OBJECTIVE. The purpose of this study was to compare the usefulness of two CT grading systems of blunt splenic trauma in predicting which patients need surgery or angioembolization.

MATERIALS AND METHODS. Four hundred patients in hemodynamically stable condition admitted with blunt splenic injury were included in the study. All patients underwent contrast-enhanced MDCT. Grade of splenic injury was prospectively assigned according to the American Association for the Surgery of Trauma (AAST) splenic injury scale. Patients were treated with surgical intervention, splenic arteriography with or without embolization, or observation alone. All MDCT images were retrospectively reviewed and regraded according to a novel grading system that specifically incorporates the findings of active bleeding or splenic vascular injury, including pseudoaneurysm and arteriovenous fistula. Receiver operating characteristics curves were generated with both grading systems for all splenic interventions, and statistical analyses were performed.

RESULTS. The area under the ROC curves for the new splenic grading system for splenic arteriography, surgery, and both interventions exceeded 80%. The area under the curve for the new splenic grading system was greater than that for the AAST injury scale for all interventions. Differences were found to be statistically significant for splenic arteriography (p = 0.0036) and the combination of arteriography and surgery (p = 0.0006).

CONCLUSION. The proposed CT grading system is better than the AAST system for predicting which patients with blunt splenic trauma need arteriography or splenic intervention.

Keywords: CT • grading system • splenic injury • trauma

Introduction

Top Abstract Introduction Materials and Methods Results Discussion References

 
Nonoperative management of blunt splenic injury is now commonly practiced [1–8]. The decision to attempt nonoperative management is largely determined by the splenic CT injury grade among other clinical factors, including patient age, presence of concurrent injuries, and the ability to perform reliable serial clinical assessments. The most widely used grading system for blunt splenic injury in trauma centers across the United States is the American Association for the Surgery of Trauma (AAST) splenic injury scale [1, 2]. This organ injury scale is based on the appearance of the spleen at surgery (Table 1). Similar CT-based grading systems, derived from the AAST scale, are based on the extent of anatomic disruption of the spleen. Previous studies [3–5] have shown that the traditional AAST injury grade and the CT-based injury grading system derived from it are poor predictors of which patients can best be treated with observation and which need angiographic or surgical intervention. The use of nonoperative management with splenic arteriography and embolization has substantial support [6–8]. Aggressive management of active splenic bleeding and vascular injuries, including pseudoaneurysm and arteriovenous fistula, with splenic artery embolization has helped to prevent failure of nonoperative management [6–9].

We have had extensive experience in the use of CT combined with arteriographic findings to identify patients most likely to need intervention for splenic injury as opposed to observation alone. We conducted a retrospective review of our experience with 400 patients to describe and compare the efficacy of two CT grading systems to optimize selection of patients for nonoperative management of blunt splenic injury to achieve a high salvage rate with minimal complications.

Materials and Methods

Top Abstract Introduction Materials and Methods Results Discussion References

 
This study was compliant with the requirements of the HIPAA and was approved by our institutional review board. Written informed consent was obtained from 76 patients. The institutional review board waived informed consent from 324 patients because this study was solely observational and included no additional interventions and because obtaining consent in these emergency and sometimes disoriented patients would have precluded the timely performance of imaging studies.

Between October 2002 and May 2005, a total of 496 patients 18 years or older were admitted to our institution with blunt splenic trauma. Patients who underwent angiography (n = 5), laparotomy (n =8), or splenectomy (n = 32) before MDCT were excluded from the study. An additional 51 patients were excluded because of death without splenic angiography or surgery within 24 hours as a result of multiple injuries (n = 10), inadequate follow-up (n = 17), presentation from an outside hospital (n = 4), delayed performance of CT (n = 2), iatrogenic splenic injury (n = 1), unavailable images (n = 7), technically inadequate CT scans (n = 9), and inadequate clinical information (n = 1). A total of 400 patients (261 men, 139 women; mean age, 38.5 years; range, 18–86 years) in hemodynamically stable condition with blunt splenic injury formed the study group. All patients underwent either 4- or 16-MDCT.

MDCT Technique
Contrast-enhanced MDCT was performed with a 4-MDCT scanner (MX 8000, Philips Medical Systems) before April 2003 and one of two 16-MDCT scanners (Brilliance 16 Power and Brilliance Big Bore, Philips Medical Systems) after April 2003, according to the techniques described in Table 2. IV and oral contrast media were administered. A biphasic IV injection of contrast material (iohexol, Omnipaque 300, GE Healthcare) was administered routinely with a power injector and a 16- to 20-gauge IV cannula. Unenhanced CT images were not obtained through the abdomen or pelvis. A 2% solution of 600 mL of meglumine diatrizoate (2% Hypaque, GE Healthcare) in water was administered orally or through a nasogastric tube if the patient was unable to drink; 300 mL was given 30 minutes before scanning and 300 mL in the scanning suite. Images of the abdomen and pelvis were acquired as part of the scan from the thoracic inlet to the symphysis pubis on the 4-MDCT scanner and as part of the whole-body scan from the circle of Willis to the symphysis pubis with the arms extended superiorly and placed next to the neck on the 16-MDCT scanner. Images of the abdomen and pelvis were acquired during the portal venous phase of contrast enhancement, and delayed images were acquired from the diaphragm to the iliac crest during the excretion phase, approximately 2–3 minutes after injection. Evaluating the spleen during these two phases was helpful in more accurately visualizing splenic vascular injuries and the renal collecting system.

Modified Grading System
The new grading system (Table 3) was based on experience from multiple trauma centers, including ours, indicating that CT evidence of active splenic hemorrhage and vascular injuries is predictive of the need for splenic arteriography and transcatheter embolization or splenic surgery [6–10]. Because of the risk of failure of nonoperative management [7–9], it is important to identify patients with these CT findings even when AAST classifications of injuries are low-grade. In the new system, such previously low-grade splenic injuries are upgraded to grade 4a or 4b. In the new system, patients with grade 4 injuries are candidates for splenic arteriography or splenic surgery. The quantity of blood in the peritoneal cavity was not taken into consideration in assigning grades with this modified system.

Definitions and Image Analysis
Initial reconstructed axial MDCT images (3- or 5-mm slice thickness) in each case were prospectively and independently reviewed by a fifth- or sixth-year resident with 1–2 years of experience or by four board-certified emergency radiologists with 3–20 years of experience. Splenic injury was graded according to the CT injury grade modeled on the AAST splenic injury scale (Table 1) [2]. Admission MDCT images in all cases were then reviewed by two radiologists who did not have access to the original interpretations or outcomes. These radiologists regraded injuries according to the modified injury grading system that specifically incorporates the findings of active bleeding or vascular lesions (Table 3).

View larger version (100K): [in this window] [in a new window] [as a PowerPoint slide]   Fig. 1A —36-year-old man with active splenic bleeding who was admitted after motor vehicle collision. Portal venous phase (A) and renal excretory phase (B) axial maximum-intensity-projection MDCT images show active bleeding (arrowheads) into peritoneum from splenic injury. Active bleeding was increased on delayed image.

View larger version (101K): [in this window] [in a new window] [as a PowerPoint slide]   Fig. 1B —36-year-old man with active splenic bleeding who was admitted after motor vehicle collision. Portal venous phase (A) and renal excretory phase (B) axial maximum-intensity-projection MDCT images show active bleeding (arrowheads) into peritoneum from splenic injury. Active bleeding was increased on delayed image.

View larger version (138K): [in this window] [in a new window] [as a PowerPoint slide]   Fig. 2A —76-year-old woman with splenic vascular injury who was admitted after motor vehicle collision. Portal venous phase (A) and renal excretory phase (B) axial MDCT images show vascular injury (arrowhead, A). Vascular injury loses density from washout of contrast material and becomes isodense with adjacent splenic parenchyma.

View larger version (126K): [in this window] [in a new window] [as a PowerPoint slide]   Fig. 2B —76-year-old woman with splenic vascular injury who was admitted after motor vehicle collision. Portal venous phase (A) and renal excretory phase (B) axial MDCT images show vascular injury (arrowhead, A). Vascular injury loses density from washout of contrast material and becomes isodense with adjacent splenic parenchyma.

Splenic Intervention
Splenic intervention was defined as splenic arteriography or surgery performed as a result of the CT-diagnosed splenic injury. Splenic arteriography was performed on all patients in hemodynamically stable condition with high-grade splenic injury (AAST grades 3–5) and on patients with low-grade splenic injury (grades 1 and 2) who had active bleeding or a vascular injury on MDCT. Splenic arteriography was performed on 164 patients. A total of 130 splenic artery embolizations were performed. Embolization was performed on 71 patients with active bleeding or vascular injury. Splenic artery embolization was performed on 59 other patients because of abrupt truncation of a splenic artery branch on arteriography or if, at the discretion of the interventional radiologist and attending trauma surgeon, the patient's clinical condition warranted a more aggressive interventional approach. For example, main splenic artery embolization was performed to control blood pressure in some patients with major head injuries in order to prevent delayed hemorrhage from a high-grade splenic injury.

At our institution, hemodynamic instability is defined with the following criteria: systolic blood pressure less than 100 mm Hg, heart rate greater than 120 beats per minute, and lack of response to a fluid challenge of 2 L of a crystalloid solution. Splenectomy was performed on 45 patients after MDCT, usually because of hemodynamic instability after CT (n = 35). Other reasons (n = 10) included injuries to adjacent organs, need for anticoagulation, high-grade splenic injury in a pregnant patient, and initiation of splenic hemorrhage when the spleen was mobilized at laparotomy.

Clinical Review and Statistical Analysis
Medical records were reviewed to determine outcome of management of splenic injuries. Statistical analysis was performed with JMP statistical software, versions 5.1 and 6.0 (SAS Institute) and SAS, version 9.1 (SAS Institute). The Stuart-Maxwell and the McNemar tests of homogeneity were used to compute statistics for assessment of the association between the old and new grading systems. To discern whether the new splenic grading system was better than the AAST injury scale in identifying patients needing splenic arteriography or surgery, receiver operating characteristic (ROC) curves were generated for both grading systems for all splenic interventions. These curves were generated for splenic arteriography and surgery separately and for all splenic interventions combined.

Nonparametric approaches were selected because they provided the foundation on which the ROC curves were based. The nonparametric method of DeLong et al. [12] was used to compare the area under the correlated AAST grade curve with that of the new-grade ROC curve from the same subjects on the basis of concepts proposed by Hanley and McNeil [13, 14]. Hanley and McNeil used Kendall's tau rank-order correlation to calculate the critical statistics for comparing the areas under two correlated ROC curves. DeLong et al. used generalized U statistics to calculate the areas under the ROC curves and to test for differences between areas under correlated ROC curves. The approach of DeLong et al. was selected because the computations yielded more precise estimates of the variance–covariance matrices and CIs of the ROC curve areas than did Hanley and McNeil's approximated estimates.

The JMP nominal logistic platform was used to generate logistic regression (logit) models for the old and new grading systems, compute the areas under the ROC curves, and produce the plots shown in Figures 3A, 3B, 4A, 4B, 5A, and 5B. The JMP logit model was reproduced with the SAS LOGISTIC procedure. The SAS %ROC macro was used to compute the 95% CIs for areas under the ROC curves and to test for significance of differences between the areas under the correlated ROC curves of the old and new grading systems.

View larger version (11K): [in this window] [in a new window] [as a PowerPoint slide]   Fig. 3A —Receiver operating characteristic (ROC) curves for arteriography and surgery combined. Diagonal dashed lines indicate 45° angle tangent line marked at point that provides best discrimination between true-positives and false-positives, assuming that false-positives and false-negatives have similar costs. Graph shows ROC curve for American Association for the Surgery of Trauma grade. Area under curve (AUC) = 0.852.

View larger version (11K): [in this window] [in a new window] [as a PowerPoint slide]   Fig. 3B —Receiver operating characteristic (ROC) curves for arteriography and surgery combined. Diagonal dashed lines indicate 45° angle tangent line marked at point that provides best discrimination between true-positives and false-positives, assuming that false-positives and false-negatives have similar costs. Graph shows ROC curve for new grading system. AUC = 0.892.

View larger version (11K): [in this window] [in a new window] [as a PowerPoint slide]   Fig. 4A —Receiver operating characteristic (ROC) curves for arteriography. Diagonal dashed lines indicate 45° angle tangent line marked at point that provides best discrimination between true-positives and false-positives, assuming that false-positives and false-negatives have similar costs. Graph shows ROC curve for American Association for the Surgery of Trauma grade. Area under curve (AUC) = 0.8199.

View larger version (11K): [in this window] [in a new window] [as a PowerPoint slide]   Fig. 4B —Receiver operating characteristic (ROC) curves for arteriography. Diagonal dashed lines indicate 45° angle tangent line marked at point that provides best discrimination between true-positives and false-positives, assuming that false-positives and false-negatives have similar costs. Graph shows ROC curve for new grading system. AUC = 0.8696.

View larger version (11K): [in this window] [in a new window] [as a PowerPoint slide]   Fig. 5A —Receiver operating characteristic (ROC) curves for surgery. Diagonal dashed lines indicate 45° angle tangent line marked at point that provides best discrimination between true-positives and false-positives, assuming that false-positives and false-negatives have similar costs. Graph shows ROC curve for American Association for the Surgery of Trauma grade. Area under curve (AUC) = 0.7918.

View larger version (11K): [in this window] [in a new window] [as a PowerPoint slide]   Fig. 5B —Receiver operating characteristic (ROC) curves for surgery. Diagonal dashed lines indicate 45° angle tangent line marked at point that provides best discrimination between true-positives and false-positives, assuming that false-positives and false-negatives have similar costs. Graph shows ROC curve for new grading system. AUC = 0.81806.

Top Abstract Introduction Materials and Methods Results Discussion References

Nonoperative management was attempted in the cases of 89% (355/400) of the patients with blunt splenic injury. Within this group, 341 (96%) of the patients were successfully treated without splenic surgery. The overall splenic salvage rate for blunt splenic injury was 85% (341/400). For high-grade injuries (grades 3–5), the success rate of nonoperative management was 95% (136/143), with an overall splenic salvage rate of 76%.

The areas under the ROC curves for the new splenic grading system for splenic arteriography, surgery, and both interventions exceeded 80%. This result means that the logistic model for the new splenic grading system had a greater than 80% chance of correctly differentiating patients needing splenic arteriography or surgery from patients who could be treated successfully with conservative management.

All areas under the ROC curves for the new splenic grading system were larger than those for the AAST injury scale (Figs. 3A, 3B, 4A, 4B, 5A, and 5B). Statistically significant differences were found between the new splenic grading system and the AAST injury scale for splenic arteriography (p = 0.0036) and for both interventions combined (p = 0.0006), according to the DeLong test results.

The larger area under the ROC curve for the new grading system suggests that use of this system provides better discriminating ability when screening patients for arteriography or surgery than does the AAST injury scale. In addition, the diagnostic accuracy of the new grading system was as effective as the AAST injury scale in screening patients for surgery (p = 0.3694).

Discussion

Top Abstract Introduction Materials and Methods Results Discussion References

 
The purpose of a grading system is to standardize reporting, plan appropriate management, and enable comparisons between institutions and studies. The AAST Organ Injury Scaling Committee was formally organized in 1987 to devise injury severity scores that could facilitate clinical investigation and outcomes research [1]. The organ injury scale is a classification scheme based on the anatomic disruption caused by injury to an individual organ. The complexity of the injury increases with the grade. The primary purpose of this injury scale is to compare outcomes of equivalent injuries managed with different protocols [1]. The organ injury scale for the spleen was revised in 1994 [2], in part as a result of the more widespread use of CT in the setting of blunt abdominal trauma and as a result of increased understanding of the relatively benign course of certain low-grade injuries. However, active bleeding and vascular injuries were not incorporated in the revised system of assigning splenic injury grades.

Although MDCT is accurate in depicting splenic injury, it has been reported [3–5] that grade of injury alone is a poor predictor of the success of nonoperative management. Treatment of patients with low-grade injuries on the AAST injury scale may fail with observation alone when vascular lesions are present and not managed appropriately. Several previous studies [6–9] have shown that the presence of splenic vascular injuries, including active bleeding, pseudoaneurysm, and arteriovenous fistula, is a predictor of failure of nonoperative management. Identification on MDCT and appropriate management of these injuries are therefore critical in achieving a high rate of success of nonoperative management. Therefore, injury grade based on the AAST injury scale cannot be used as the sole criterion for guiding management.

Unlike the AAST injury scale, the new CT-based grading system we propose incorporates MDCT findings of vascular lesions and active bleeding in assigning the splenic injury grade. In this grading scheme, all patients with active bleeding or vascular injury are considered to have grade 4 injury. Patients with grade 4 injury would need splenic angiography and transcatheter embolization or splenic surgery. Our findings showed that in the new grading system, the injuries of 54 patients would have been upgraded to grade 4 from the grade initially assigned with the AAST injury scale. Even with our aggressive institutional policy of performing splenic arteriography on all patients with splenic vascular injury or high-grade splenic injury (AAST grades 3–5), 20 patients would have been treated without angiography or surgery because their AAST-classified injuries were low-grade (AAST grades 1 and 2), yet these patients clearly were at risk of failure of nonoperative management. In our study, splenic intervention (splenic arteriography and transcatheter embolization for vascular injury in 80% of cases and splenectomy in 10%) was needed by 90% of patients with splenic injuries upgraded from low-grade.

The aim of the proposed grading system is to identify cases in which observation alone is likely to fail. Thompson et al. [10] reported using three CT findings that correlated with the need for intervention: a large amount of hemoperitoneum, lacerations or devascularization involving more than 50% of the surface area of the splenic parenchyma, and a contrast blush larger than 1 cm. As those authors indicated, CT findings were validated in only a small number of patients. Only eight of the 56 patients in that study needed splenic intervention. In our study, 209 splenic interventions were performed in 392 patients with blunt splenic injury, a volume of studies that helped us to validate the new grading system. Our new grading system does not take into consideration the amount of blood in the peritoneal cavity. A prospective study is necessary to determine whether the new grading system enables optimal prediction of the success of nonoperative management of blunt splenic injuries.

In this study, we used areas under ROC curves to compare the AAST and new grading systems. In medical decision making, ROC curves have been used to measure the accuracy of diagnostic tests in discriminating between disease and nondisease, or, in our case, between patients who did and did not need splenic intervention [15]. An ROC curve is a plot of the true-positive rate versus the false-positive rate (i.e., sensitivity vs 1 – specificity over the full range of possible test outcome thresholds). The area under the ROC curve is a commonly used index for summarizing test accuracy [16]. Areas under ROC curves vary between 0.5 and 1.0, a curve area of 1.0 representing perfect test accuracy. The increase in the area under the ROC curves (overall, 0.85–0.89) in this study shows that the new grading system is better for prediction of the need for splenic intervention in blunt splenic injury. A larger increase was observed for arteriography alone than for surgery (0.82–0.87 and 0.79–0.82, respectively). This result may have occurred because the decision to proceed to surgery is multifactorial, including clinical and radiologic findings. For a large proportion of the surgical patients (35 of 45), the reason for laparotomy was hemodynamic instability rather than MDCT findings alone.

The method for determining whether the difference in the area under the two ROC curves is statistically significant was based on a nonparametric method. Hanley and McNeil [14] originally described a parametric method for two curves derived from the same cases, but estimates rather than exact values were used. For this reason, we used the approach of DeLong et al. [12] and found statistically significant differences between the areas under the ROC curves for arteriography and the combination of arteriography and surgery. The increase in area under the curve for surgery from the AAST scale to the new grading scale was not found to be statistically significant.

This study had several limitations. It was retrospective and performed at a single institution with current department protocols for selecting patients for splenic intervention. Obuchowski et al. [17] cited the use of nonparametric 95% CIs as being too narrow. However, the parametric properties of the ROC curve are not well established, and studies are being conducted to investigate ways to overcome these limitations. Interobserver and intraobserver variabilities in use of the new grading system were not determined in our study. Verifying the superiority of the new grading system over the AAST injury scale requires either a prospective investigation or a retrospective follow-up study with a different sample of representative cases. We are undertaking bootstrap resampling approaches to simulate, model, and perform the prospective analysis described previously [11].

The results of our retrospective study with a large number of subjects in which the AAST splenic injury grading system was compared with a proposed MDCT-based splenic injury grading system that incorporates assessments of active bleeding and splenic vascular injury indicate the new grading system is helpful for triage of patients who need splenic arteriography and surgery. A prospective multicenter study is needed to validate these results.

Acknowledgments
 
We thank Nancy Knight for assistance in the preparation of this manuscript for publication.

References

Top Abstract Introduction Materials and Methods Results Discussion References

 

1. Moore EE, Shackford SR, Pachter HL, et al. Organ injury scaling: spleen, liver and kidney. J Trauma 1989;29 : 1664-1666[Medline]
2. Moore EE, Cogbill TH, Jurkovich GJ, Shackford SR, Malangoni MA, Champion HR. Organ injury scale: spleen and liver (1994 revision). J Trauma 1995; 38:323 -324[Medline]
3. Becker CD, Spring P, Glattli A, Schwiexer W. Blunt splenic trauma in adults: can CT findings be used to determine the need for surgery? AJR 1994; 162:343 -347[Abstract/Free Full Text]
4. Sutyak JP, Chiu WC, D'Aelio LF, Amorosa JK, Hammond JS. Computed tomography is inaccurate in estimating the severity of adult splenic trauma. J Trauma 1995; 39:514 -518[Medline]
5. Kohn JS, Clark D, Isler RJ, Pope CF. Is CT grading of splenic injury useful in the nonsurgical management of blunt trauma? J Trauma 1994; 36:385 -390[Medline]
6. Shanmugathan K, Mirvis SE, Boyd-Kranis R, Takada T, Scalea TM. Nonsurgical management of blunt splenic trauma: use of CT criteria to select patients for splenic arteriography and potential endovascular therapy. Radiology 2000;217 : 75-82[Abstract/Free Full Text]
7. Federle MP, Courcoulas AP, Powell M, Ferris JV, Peitzman AB. Blunt splenic injury in adults: clinical and CT criteria for management, with emphasis on active extravasation. Radiology1998; 206:137 -142[Abstract/Free Full Text]
8. Schurr MJ, Fabian TC, Gavant ML, et al. Management of blunt splenic trauma: computed tomographic contrast blush predicts failure of nonoperative management. J Trauma 1995;39 : 507-512[Medline]
9. Gavant ML, Schurr MJ, Flick PA, et al. Predicting clinical outcome of nonsurgical management of blunt splenic injury: using CT to reveal abnormalities of splenic vasculature. AJR1997; 168:207 -212[Abstract/Free Full Text]
10. Thompson EE, Munera F, Cohen SM, et al. Computed tomography scan scoring system predicts the need for intervention after splenic injury. J Trauma 2006; 60:1083 -1086[Medline]
11. Shanmuganathan K, Mirvis SE, Sover ER. Value of contrast-enhanced CT in detecting active hemorrhage in patients with blunt abdominal or pelvic trauma. AJR 1993;161 : 65-69[Abstract/Free Full Text]
12. DeLong ER, DeLong DM, Clark-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1998;44 : 837-845[CrossRef]
13. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143 : 29-36[Abstract/Free Full Text]
14. Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983;148 : 839-843[Abstract/Free Full Text]
15. Lusted LB. Signal detectability and medical decision making. Science 1971; 171:1217 -1219[Free Full Text]
16. Centor RM. Signal detectability: the use of ROC curves and their analyses. Med Decis Making 199111 : 102-106[Abstract/Free Full Text]
17. Obuchowski NA, Beiden SV, Berbaum KS, et al. Multireader, multicase receiver operating characteristic analysis: an empirical comparison of five methods. Acad Radiol 2004;11 : 980-995[Medline]

CiteULike

Complore

Connotea

Del.icio.us

Digg

Reddit

Technorati

This Article Abstract Figures Only Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Marmery, H. Articles by Mirvis, S. E. Search for Related Content PubMed PubMed Citation Articles by Marmery, H. Articles by Mirvis, S. E. Social Bookmarking                      What's this? Hotlight (NEW!) What's Hotlight?