June 2009, VOLUME 192
NUMBER 6

Recommend & Share

June 2009, Volume 192, Number 6

Health Care Policy and Quality

Perspective

Do Clinicians Use the American College of Radiology Appropriateness Criteria in the Management of Their Patients?

+ Affiliation:
1All authors: Department of Radiology, Robert Wood Johnson University Hospital and Robert Wood Johnson Medical School/UMDNJ, 1 Robert Wood Johnson Pl., New Brunswick, NJ 08901.

Citation: American Journal of Roentgenology. 2009;192: 1581-1585. 10.2214/AJR.08.1622

ABSTRACT
Next section

OBJECTIVE. The purpose of this study was to investigate the use of the American College of Radiology (ACR) appropriateness criteria by referring physicians during decision making about imaging in the management of their patients.

CONCLUSION. There is a low utilization of the ACR appropriateness criteria by clinicians when ordering imaging studies for their patients. The ACR has invested a great deal of resources in these criteria and should therefore be aware of information regarding utilization. Our findings may have implications about how the ACR appropriateness criteria are reviewed, revised, and disseminated.

Keywords: ACR appropriateness criteria, diagnostic imaging, patient management, practice of radiology, referring physicians

Introduction
Previous sectionNext section

According to the American College of Radiology (ACR), imaging is one of the fastest-growing services in medicine, with costs reaching approximately $100 billion annually [1]. Recent technologic advances in CT, ultrasound, and MRI have rapidly increased the utilization of these techniques by physicians, a trend that showed great growth in the 1990s and that continues in the 21st century [2]. The clinical information acquired from the procedures, the decrease in time needed to perform them, and greater accessibility to imaging facilities have made diagnostic imaging more attractive to both patients and referring physicians.

However, despite the potential benefits of increased utilization, one must not discount the medical and economic risks associated with diagnostic imaging. Recent articles such as that by Brenner and Hall [3] have generated considerable public scrutiny and awareness regarding CT and its associated radiation exposure and carcinogenic potential. There is a growing concern about the possibility of contrast-related reactions ranging from mild allergies to severe reactions such as gadolinium-related nephrogenic systemic fibrosis [4]. Given that health care expenditures in the United States are expected to grow from 16% of the gross domestic product in 2007 to 20% of the gross domestic product in 2016—more than in any other industrialized nation—regulating health care costs including the costs of imaging is essential [5].

Referring physicians need to select the appropriate imaging technique to ensure cost-effective, high-quality patient care [6]. In 1993, the ACR developed the ACR appropriateness criteria [7], scientific-based guidelines intended to guide referring physicians about the favorable use of diagnostic and interventional radiology for given clinical situations. Fifteen years since its inception, our study assesses the progress of the ACR in promoting this initiative and the prevalence of ACR appropriateness criteria use by referring physicians when ordering imaging examinations for their patients. It is in the best interest of patients, referring physicians, health care payers, and radiologists that imaging is used appropriately [8].

Materials and Methods
Previous sectionNext section

This study was approved by our local institutional review board for exemption. A survey using the Website www.surveymonkey.com was created to investigate the common resources that physicians at our 500-bed university medical center use when determining the proper imaging tech nique for a patient's clinical problem. Our survey had two components: general demographic information about the physicians and specific questions (Fig. 1). Physicians were asked to se lect their top three choices from the following 11 choices: Google [9], recent continuing medial education (CME) meetings, your specialty jour nal, UpToDate [10] (an evidence-based, peer-reviewed medical information resource), ACR appropri ateness criteria, MD Consult, PubMed [11], radiologist consult, a book from the Pocket Medicine series [12], or a book from the Washington Manual series [13], and an Other option with the opportunity to enter a source not listed.

The physicians surveyed included both general and subspecialists of internal medicine, surgery, obstetrics and gynecology, pediatrics, family medicine, psychiatry, and neurology. The survey was not administered to emergency department physicians because prior studies have shown that up to 70% of emergency physicians admit to ordering more diagnostic tests than are medically indicated due to the current era of defensive medicine [14]. An explanation of the purpose of the survey and a request for responses were sent by e-mail to staff physicians on behalf of a senior radiologist of the hospital. Anonymous responses were collected during a 14-day period. The e-mail was re-sent to staff physicians after the first 7 days to encourage those who had not yet completed the survey to respond.

Results
Previous sectionNext section
Demographics

Of the 438 physicians, 201 residents and 237 attending physicians, who were e-mailed the survey, 126 physicians (28.8%), 59 residents and 67 attending physicians, completed it. The largest number of responses came from the internal medicine (n = 57), surgery (n = 25), and pediatrics (n = 24) departments (Table 1).

TABLE 1: Demographics of Survey Respondents

Survey Results

Two of the 126 physicians (1.59%), one first-year postgraduate internal medicine resident and one pediatrics attending physician, who completed our survey claim to use the ACR appropriateness criteria as the first source when selecting the best imaging technique for their patients (Fig. 2). Radiologist consult (n = 29), UpToDate (n = 24), and your specialty journal (n = 24) were the most frequently used resources. Physicians who selected Other and opted to choose a resource that was not listed as an available choice cited personal experience (n = 7) and asking a fellow colleague in their specialty (n = 4) as the most important tools in assisting them in their imaging decision making.

One of the 126 physicians (0.79%), a neurology attending physician, who completed our survey claims to use the ACR appropriateness criteria as the second reference when selecting the best imaging technique for patients (Fig. 3). Zero of the total 126 physicians who completed our survey claim to use the ACR appropriateness criteria as the third source when selecting the best imaging technique (Fig. 4). Many physicians chose Google (n = 20) as a third source to assist them in imaging decision making.

figure
View larger version (37K)
Fig. 1 Original survey that was sent to 438 physicians at our institution. One hundred twenty-six physicians responded. PGY = postgraduate year, CME = continuing medical education, ACR = American College of Radiology.

UpToDate (n = 37, 62.7%), radiologist consult (n = 31, 52.5%), and Google (n = 27, 45.8%) were the resources that were most frequently ranked as one of the top three by the resident physicians who responded to our survey. Radiologist consult (n = 50, 74.6%), your specialty journal (n = 47, 70.1%), and recent CME meetings (n = 20, 29.9%) were the resources that were most frequently ranked as one of the top three by the attending physicians who responded to our survey. Compiling the responses of both resident and attending physicians overall, radiologist consult (n = 81, 64.3%), your specialty journal (n = 61, 48.4%), UpToDate (n = 52, 41.3%), and Google (n = 35, 27.8%) were the resources that were most frequently ranked as one of the top three (Table 2).

TABLE 2: Resources Used by Referring Physicians in Choosing the Most Appropriate Imaging Technique to Order for Their Patients

Discussion
Previous sectionNext section

The ACR appropriateness criteria are evidence-based guidelines designed to assist referring clinicians in making the most appropriate decision about imaging or treatment for given clinical situations. Significant resources have been invested in both developing these criteria and ensuring the criteria are up-to-date, reflecting recent changes in technology and clinical practice. The guidelines are developed by expert panels in diagnostic imaging, interventional radiology, and radiation oncology. Each panel includes leaders in radiology and other specialties. The criteria currently include more than 160 topics with over 700 variants [7].

The need to inform referring clinicians about the rapidly changing field of imaging has been recognized for a long time [15]. In the culture of radiology, radiologists are a service provider and consultant to other departments, and it is common practice to simply perform the procedure requested by the clinical team. Referring clinicians are not accustomed to having their orders for procedures rejected. Furthermore, such circumstances could potentially result in unpleasant encounters between the clinician and radiologist. Aside from the absence of a reimbursable ICD-9 code [16], most imaging centers do not have strict criteria to reject unnecessary or inappropriate requests for imaging. This criterion also may be inadequate considering that there are generally no shortages of ICD-9 codes that can be marginally linked to a clinical scenario.

figure
View larger version (14K)
Fig. 2 Bar graph shows number of physicians who reported using resource as first resource in choosing most appropriate imaging technique for patients. CME = continuing medical education, ACR = American College of Radiology.

figure
View larger version (14K)
Fig. 3 Bar graph shows number of physicians who reported using resource as second resource in choosing most appropriate imaging technique for patients. CME = continuing medical education, ACR = American College of Radiology.

The results of studies suggest a general lack of awareness among clinicians about the appropriateness of imaging studies and that perhaps the current methods of choosing these procedures are inadequate. For example, Taragin et al. [17] administered a survey in which medical house staff were asked to choose the appropriate imaging examinations for specific clinical situations and found that fewer than 50% of the respondents were able to answer half of the questions correctly. These findings suggest that the house staff are not adequately prepared in this proficiency. With these considerations, there is a general sense among clinicians that a set of guidelines would be helpful and more effective in assessing the appropriateness of certain imaging examinations compared with the resources that are currently being used.

figure
View larger version (14K)
Fig. 4 Bar graph shows number of physicians who reported using resource as third resource in choosing most appropriate imaging technique for patients. CME = continuing medical education, ACR = American College of Radiology.

Levy et al. [18] reported that the use of the ACR appropriateness criteria by general practitioners in a preauthorization center led to better utilization of MRI, an increase in the rate of performance of appropriate MRI examinations, and a decrease in the rate of performance of inappropriate MRI examinations. As similar studies gain more recognition along with the current efforts to curb increasing health care expenditures, the ACR appropriateness criteria will become an invaluable tool to both referring physicians and radiologists. Several large private payers have already begun to establish preauthorization schemes for imaging, and reimbursement policies have become more stringent [19]. The ACR appropriateness criteria can influence clinical decision making toward ordering the most appropriate and cost-effective imaging studies for patients.

Our study shows significantly low utilization of the ACR appropriateness criteria: three of 126 physicians (2.4%). We suspect that there is a general lack of awareness about the existence of these criteria. In fact, one of the authors was not aware of the ACR appropriateness criteria before getting involved in this project. Therefore, it is not a surprise that more popular and widely accessible resources such as UpToDate, MD Consult, PubMed, and a physician's specialty journal are preferred. Interestingly, 35 of the 126 physicians (27.8%) selected Google as one of their top three resources, an especially popular choice among resident respondents. The fact that this group would rather turn to a random Internet search engine indicates a great need for awareness of the more comprehensive, evidence-based, and consensus opinion–derived ACR appropriateness criteria. We believe that further information about the responding physicians' awareness, opinions, or usage of the ACR appropriateness criteria could not be obtained from our survey without asking additional questions that would have biased the results.

Eighty-one of 126 physicians (64%) indicated that they consult a radiologist as one of their top three resources, the most popular choice by both residents and attending physicians. Radiologists do welcome consults with referring physicians, which aids them in ordering the most appropriate study and obtaining invaluable clinical information that could provide a more accurate and clinically applicable interpretation of the study. These consultations traditionally result in optimal patient care and should be implemented when possible. However, in cases in which referring clinicians do not have the opportunity to consult with a radiologist for whatever reason, they may benefit from referring to the ACR appropriateness criteria as a reliable second option. The ACR strongly promotes this resource because it is practical to use, scientifically accurate, and derived from an evidence-based and consensus methodology [6].

Use of the ACR appropriateness criteria can potentially improve referring clinicians' decision making about imaging and may serve as a starting point for clinical discussion if the criteria are reviewed by the ordering physician before or during the radiologist consult. In other cases, it may perhaps save the clinician a trip to the reading room or a telephone call to the radiologist regarding uncomplicated and standard clinical situations that are covered by the guidelines, thus allowing improved radiologist efficiency. In an age of decreasing reimbursement for medical services and pay for performance, increased efficiency and increased productivity of a health care professional are of paramount importance [20].

We think that the survey respondents are a fair representation across various specialties in our study. A potential limitation that warrants consideration is that we did not include emergency department physicians, arguably one of the largest consumers of medical imaging. However, there are fundamental differences in the setting in which emergency medicine is practiced compared with other specialties. The lack of a comprehensive history and physical, the acuity of a patient's clinical condition, lack of access to certain tests, and medical–legal restrictions dictate the behavior of an emergency medicine physician in a unique way. Studdert et al. [14] found that up to 70% of the emergency physicians who were surveyed for their study reported ordering CT, MRI, or radiography examinations that were not clinically necessary because of the threat of malpractice liability. We chose to exclude emergency medicine physicians to avoid any confounding findings.

Personal experience as a potential resource for imaging decisions was not included as one of the original survey choices that respondents could select. We thought it was implied that personal experience is used by all referring clinicians when ordering imaging examinations, and we were interested in investigating which resources they use when personal experience is not sufficient to render a decision. Therefore, the seven physicians (5.6%) who indicated personal experience by entering it under the Other option are a clear underestimation.

Considering the resources that were most frequently used by physicians in our study, an ideal reference would be one that is valid, widely recognized, user-friendly, and accessible. The ACR continues to invest a great deal to ensure the validity of its guidelines by continuing to regularly review the appropriateness criteria to be consistent with the most recent scientific data and to make additions that reflect new technology and trends in clinical practice, with the ultimate goal of improving patient care.

The finding in our study of low utilization of the ACR appropriateness criteria by referring physicians may serve as a starting point for the ACR to reassess the appropriateness criteria and identify strategies for better disseminating the information. The 2006 request for proposals for ACR appropriateness criteria use in medical education [21] and similar innovations by radiology educators have been implemented in the past to accomplish this goal. It may be worthwhile to promote the ACR appropriateness criteria through the primary sources that referring clinicians commonly use now as suggested by our study: through radiologist consults, specialty journals, and Google. Improving access to the criteria and incorporating them into a search engine database may make them more user-friendly and comparable to widely used evidence-based medicine databases such as UpToDate. Hospital computer systems could hyperlink to or incorporate the ACR appropriateness criteria, so with every order the clinician has the opportunity to use the criteria by default or to select an examination based on the criteria.

If medical students and residents can be taught to use and rely on the ACR appropriateness criteria during their training, they will be more likely to refer to these guidelines throughout their careers [3]. This will ensure that future referring physicians will use imaging effectively and efficiently, which is the central goal of the ACR appropriateness criteria.

Address correspondence to J. K. Amorosa ().

References
Previous sectionNext section
1. Mishori R. The danger of too many tests. Parade. July 6, 2008. www.parade.com/articles/editions/2008/edition_07-06-2008/3Too_Many_Tests. Accessed July 7, 2008 [Google Scholar]
2. Bhargavan M, Sunshine JH. Utilization of radiology services in the United States: levels and trends in modalities, regions, and populations. Radiology 2005; 187:824–832 [Google Scholar]
3. Brenner DJ, Hall EJ. Computed tomography: an increasing source of radiation exposure. N Engl J Med 2007; 357:2277 –2284 [Google Scholar]
4. Wiginton CD, Kelly B, Oto A, et al. Gadolinium-based contrast exposure, nephrogenic systemic fibrosis, and gadolinium detection in tissue. AJR 2008; 190:1060 –1068 [Abstract] [Google Scholar]
5. National Coalition on Health Care Website. Health insurance costs: facts on the cost of health insurance and health care. www.nchc.org/facts/cost.shtml. Accessed July 13, 2008 [Google Scholar]
6. Bettmann M. The ACR appropriateness criteria: view from the committee chair. J Am Coll Radiol 2006; 3:510–512 [Google Scholar]
7. American College of Radiology Website. ACR appropriateness criteria: background and development. www.acr.org/SecondaryMainMenuCategories/quality_safety/app_criteria/BackgroundandDevelopment.aspx. Accessed March 9, 2008 [Google Scholar]
8. Gunderman RB, Bettmann M, Davis LP. Promoting educational innovation: lessons from the request for proposals for ACR appropriateness criteria usage in medical education. J Am Coll Radiol 2007; 4:919 –924 [Google Scholar]
9. Google search engine. Mountain View, CA. www.google.com [Google Scholar]
10. UpToDate [database online]. Waltham, MA. www.uptodate.com/home/index.html [Google Scholar]
11. U.S. National Library of Medicine, National Institutes of Health. PubMed [database online]. Bethesda, MD. www.ncbi.nlm.nih.gov/sites/ [Google Scholar]
12. Sabatine SA, ed. Pocket medicine: the Massachusetts General Hospital handbook of internal medicine, 3rd ed. Pocket notebook series. Philadelphia, PA: Lippincott Williams & Wilkins,2008 [Google Scholar]
13. Cislo GS, De Fer TM, eds. The Washington manual general internal medicine consult, 2nd ed. Washington manual series. Philadelphia, PA: Wolters Kluwer/Lippincott Williams & Wilkins,2009 [Google Scholar]
14. Studdert DM, Mello MM, Sage WM, et al. Defensive medicine among high-risk specialist physicians in a volatile malpractice environment. JAMA 2005; 293:2609 –2617 [Google Scholar]
15. Amorosa JK, Nosher JL, Amorosa LF. Updating one's colleagues. (editorial) AJR 1981; 137:423 [Google Scholar]
16. Hamm WK, Jackson JR, Morris RD, et al., eds. International classification of diseases, 9th rev. Montgomery, AL: Unicor Medical, 1995 [Google Scholar]
17. Taragin BH, Feng L, Ruzal-Shapiro C. Online radiology appropriateness survey: results and conclusions from an academic internal medicine residency. Acad Radiol 2003; 10:781–785 [Google Scholar]
18. Levy G, Blachar A, Goldstein L, et al. Nonradiologist utilization of ACR appropriateness criteria in a preauthorization center for MRI requests: applicability and effects. AJR 2006; 187:855–858 [Abstract] [Google Scholar]
19. Sistrom CL. In support of the ACR appropriateness criteria. J Am Coll Radiol 2008; 5:630–635 [Google Scholar]
20. Moser JW, Wilcox PA, Bjork SS, et al. Pay for performance in radiology: ACR white paper. J Am Coll Radiol 2006; 3:650 –664 [Google Scholar]
21. American College of Radiology Website. ACR issues request for proposals for ACR appropriateness criteria usage in medical education. www.acr.org/SecondaryMainMenuCategories/NewsPublications/FeaturedCategories/CurrentACRNews/archive/1262006ACRIssuesRequestforProposalsforACRAppropriatenessCriteriaUsageinMedicalEducationDoc46.aspx. Published December 6, 2006. Accessed March 2, 2009 [Google Scholar]
22. Epocrates PDA medical software. San Mateo, CA. www.epocrates.com [Google Scholar]

Recommended Articles

Do Clinicians Use the American College of Radiology Appropriateness Criteria in the Management of Their Patients?

Full Access, ,
American Journal of Roentgenology. 2006;186:937-942. 10.2214/AJR.05.0041
Abstract | Full Text | PDF (277 KB) | PDF Plus (367 KB) 
Full Access, , , , , , ,
American Journal of Roentgenology. 2006;187:855-858. 10.2214/AJR.05.1055
Abstract | Full Text | PDF (220 KB) | PDF Plus (295 KB) 
Full Access, ,
American Journal of Roentgenology. 2013;201:W619-W625. 10.2214/AJR.12.10487
Abstract | Full Text | PDF (593 KB) | PDF Plus (643 KB) 
Full Access
American Journal of Roentgenology. 2009;192:W330-W334. 10.2214/AJR.08.2190
Abstract | Full Text | PDF (806 KB) | PDF Plus (769 KB) 
Full Access, , , , , , , , , , , ,
American Journal of Roentgenology. 2019;212:859-866. 10.2214/AJR.18.19931
Abstract | Full Text | PDF (707 KB) | PDF Plus (766 KB) 
Full Access, , , , , , , , , , , ,
American Journal of Roentgenology. 2002;178:1335-1347. 10.2214/ajr.178.6.1781335
Citation | Full Text | PDF (4695 KB) | PDF Plus (4831 KB)