HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Sohlich, R. E. Articles by Dee, K. E. Search for Related Content PubMed PubMed Citation Articles by Sohlich, R. E. Articles by Dee, K. E. Social Bookmarking                      What's this?

Interpreting Data from Audits When Screening and Diagnostic Mammography Outcomes Are Combined

¹ Department of Radiology, Box 1667, University of California Medical Center, San Francisco, CA 94143-1667.
² Department of Radiology, Box 357115, University of Washington Medical Center, 1959 N.E. Pacific St., Seattle, WA 98195.

Received June 28, 2001; accepted after revision September 6, 2001.

 
Presented in part at the annual meeting of the American Roentgen Ray Society, Seattle, April—May 2001.

Address correspondence to E. A. Sickles.

Abstract

Top Abstract Introduction Materials and Methods Results Discussion References

 
OBJECTIVE. The objective of this study was to use mathematic models to aid mammography practices in interpreting outcomes data derived from a combination of screening and diagnostic examinations, and in interpreting diagnostic mammography outcomes data that are not segregated by indication for examination.

MATERIALS AND METHODS. We analyzed outcomes from 51,805 consecutive mammography examinations. Screening and diagnostic examinations were audited separately. Diagnostic examinations were audited by indication for examination. Extrapolating from our known mix of screening (79%) and diagnostic (21%) examinations, we determined expected combined outcomes for various mixes that might be encountered in clinical practice. Similarly, we determined the expected overall diagnostic mammography outcomes for various clinically relevant mixes of indications for examination.

RESULTS. Outcomes vary substantially depending on the mix of screening and diagnostic examinations performed. For example, expected outcomes for practices with screening—diagnostic mixes of 90-10% and 50-50% are, respectively: rate of abnormal findings, 6% versus 11%; rate of positive biopsy findings, 38% versus 42%; cancer detection rate, 10 per 1,000 versus 30 per 1,000; mean invasive cancer size, 14.4 mm versus 16.0 mm; nodal metastasis rate, 8% versus 11%; and rate of stage 0 and stage I cancers, 87% versus 82%. Diagnostic outcomes also vary substantially according to indication for examination, with a higher rate of abnormal findings, a higher rate of positive biopsy findings, and a larger mean invasive cancer size expected for mixes involving a high percentage of workups for palpable lesions.

CONCLUSION. When screening and diagnostic mammography outcomes are not segregated during auditing, and when diagnostic outcomes are not segregated by indication for examination, analysis of combined audit data should be based on extrapolations from known outcomes.

Introduction

Top Abstract Introduction Materials and Methods Results Discussion References

 
To comply with federal regulations based on the Mammography Quality Standards Act, all mammography facilities in the United States must perform limited medical outcomes audits [1]. Optional, more comprehensive audits, described by the American College of Radiology in its Breast Imaging Reporting and Data System (BI-RADS) [2], are valuable tools to measure the success of a mammography practice in detecting clinically occult, early-stage breast cancer, as well as to suggest the presence of any deficiencies in technical performance and image interpretation [2,3,4,5,6].

On the basis of published mammography outcomes data derived from large screening practices and population-based screening programs [3,4,5, 7,8,9,10,11,12,13], desirable goals have been put forth for the detection of breast cancer in asymptomatic women [2, 6, 9, 14]. However, substantial differences in mammography outcomes are found when auditing screening versus diagnostic examinations [15, 16], and some of these differences have been shown to be statistically significant [16]. The typical mammography practice involves a mixture of screening and diagnostic examinations, but the percentage of screening versus diagnostic examinations may vary considerably. Therefore, a mammography practice should expect its own outcomes data to vary depending on its specific mix of cases.

Diagnostic mammography examinations are performed for a variety of problem-solving indications ranging from the presence of breast implants in asymptomatic women to the evaluation of the extent of a known breast cancer. Insofar as the probability of malignancy is vastly different in these clinical scenarios, substantial differences in mammography outcomes are found when diagnostic examinations themselves are segregated by indication for examination [16]. Again, a mammography practice should expect its own diagnostic outcomes data to vary depending on its specific mix of indications for examination.

Although subset analyses of auditing data (e.g., screening versus diagnostic examinations, diagnostic examinations as a function of indication for examination) produce more meaningful results, federal regulations do not mandate this type of auditing, and the time and cost involved are substantial. These factors limit the widespread use of comprehensive auditing. In this article, we present data to aid mammography facilities that do not segregate mammography outcomes during auditing. By developing mathematic models based on outcomes data from our own comprehensively audited practice, we have constructed tables of expected outcomes for the full range of case mixes encountered in mainstream clinical practice.

Materials and Methods

Top Abstract Introduction Materials and Methods Results Discussion References

 
From January 1997 through January 2001, data were collected for all 51,805 screening and diagnostic mammography examinations that were performed at fixed-site facilities in our institution. Some of these data have been reported previously [16]. Screening examinations involved asymptomatic women, consisted of mediolateral oblique and craniocaudal projections of each breast, and were interpreted in batches twice daily. We performed diagnostic mammography as a problem-solving examination that used the full spectrum of mammographic projections, as indicated for each case. Diagnostic examinations were interpreted immediately after imaging was completed. Data for both screening and diagnostic examinations were stored in a computer database that was patterned on principles described previously [17].

We segregated our diagnostic examinations according to indication for examination using the following categories: workup of a screening examination with abnormal findings (20% of cases), short-interval follow-up of a probably benign lesion (6%), periodic surveillance of a cancer patient treated with breast preservation surgery (lumpectomy) (20%), workup of a palpable mass (15%), and other (39%). The "other" category was used for examinations that did not fit into any of the previous categories and included some asymptomatic women with breast implants or a history of previous mastectomy for carcinoma, women with a history of focal breast pain or palpable breast thickening, and women with known breast cancer being examined either for extent of disease before surgery or for response to neoadjuvant chemotherapy.

At the time of image interpretation, the radiologist entered in our computer database the indication for the examination, a BI-RADS assessment category for each breast, and management recommendations. Subsequently, we recorded a benign or malignant diagnosis for all cases in which biopsy was performed. For biopsies showing malignancy (defined as ductal carcinoma in situ or any invasive carcinoma), we also recorded the size, nodal status, and stage of the cancer, based on the American Joint Committee on Cancer staging system [18].

On the basis of these data, we performed a medical audit of our practice, segregating screening from diagnostic examinations and subdividing the diagnostic examinations into categories according to indication for examination. In this manner, we were able to perform subset analyses that we then used as the source for spreadsheet-based mathematic (simple linear) extrapolations of outcomes data involving a series of case mixes of screening versus diagnostic examinations, and a series of case mixes involving different percentages of indications for diagnostic examinations.

Results

Top Abstract Introduction Materials and Methods Results Discussion References

 
Screening Versus Diagnostic Outcomes
This study involved 40,691 screening examinations and 11,114 diagnostic examinations. Among the screening examinations, 2,110 were interpreted as having abnormal findings (BI-RADS assessment category 0, 4, or 5). Recommended biopsies eventually were performed for 572 of these patients, yielding a diagnosis of malignancy in 219 cases. Invasive cancer was found in 140 cases, 10 of which were axillary nodal metastasis. For all the screening-detected cancers, 194 were stage 0 or stage I.

Among the diagnostic examinations, 1,779 were interpreted as having abnormal findings (BI-RADS assessment category 4 or 5). Recommended biopsies were performed for 1,301 of these examinations, yielding a diagnosis of malignancy in 600 cases. Invasive cancer was found in 416 cases, 84 of which were axillary nodal metastasis. For all cancers detected at diagnostic mammography, 450 were stage 0 or stage I.

Table 1 summarizes the outcomes data for screening and diagnostic examinations in our practice. Based on mathematic extrapolation using these observed data, Table 2 displays combined screening plus diagnostic outcomes data derived for a range of case mixes of screening and diagnostic examinations that may be encountered in clinical practice.

We designed the data in Table 2 to be used by mammography practices that do not segregate screening and diagnostic examinations during auditing. If a given practice can determine (or estimate) its own case mix of screening and diagnostic examinations, it can compare its observed outcomes data with those presented in the table. For example, a practice that involves 30% screening and 70% diagnostic mammography should expect to observe different outcomes from those of a practice that involves 90% screening and 10% diagnostic mammography. For these two respective practices, the estimated rates of abnormal findings are 13% versus 6%, the estimated rates of positive biopsy findings are 44% versus 38%, the estimated cancer detection rates are 39 per 1,000 versus 10 per 1,000, the estimated rates of axillary nodal metastasis are 12% versus 8%, the estimated rates of stage 0 and stage I cancers are 79% versus 87%, and the estimated mean sizes of invasive cancer are 16.8 mm versus 14.4 mm.

Diagnostic Outcomes as a Function of Indication for Examination
This analysis involved 11,114 diagnostic examinations. Of these cases, 2,214 were performed for the workup of abnormal screening examinations, 673 for short-interval follow-up of probably benign lesions, 2,253 for short-interval follow-up of lumpectomy patients, 1,692 for the evaluation of palpable masses, and 4,282 for other (miscellaneous) reasons. Among the diagnostic examinations performed to work up abnormal screening examinations, 814 were interpreted as abnormal (biopsy recommended). Biopsies eventually were performed for 640 of these examinations, yielding a diagnosis of malignancy in 248 cases. Invasive cancer was found in 147 cases, 13 of which were axillary nodal metastasis. For all the detected cancers, 217 were stage 0 or stage I. Among the diagnostic examinations performed for short-interval follow-up of probably benign lesions, 43 were interpreted as abnormal (biopsy recommended). Biopsies were performed for 30, yielding a diagnosis of malignancy in three cases. Invasive cancer was found in two cases, neither of which was axillary nodal metastasis. All three cancers were stage 0 or stage I. Among the diagnostic examinations performed for short-interval follow-up of lumpectomy patients, 63 were interpreted as abnormal (biopsy recommended). Biopsies were performed for 49, yielding a diagnosis of malignancy in 36 cases. Invasive cancer was found in 21 cases, one of which had axillary nodal metastasis. For all the detected cancers, 32 were stage 0 or stage I. Among the diagnostic examinations performed to evaluate a palpable mass, 385 were interpreted as abnormal (biopsy recommended), and biopsies were performed for 337, yielding a diagnosis of malignancy in 209 cases. Invasive cancer was found in 179 cases, 58 of which were axillary nodal metastasis. For all the detected cancers, 101 were stage 0 or stage I. Among the diagnostic examinations performed for other reasons, 474 were interpreted as abnormal (biopsy recommended), and biopsies were performed for 298, yielding a diagnosis of malignancy in 126 cases. Invasive cancer was found in 67 cases, 12 of which were axillary nodal metastasis. For all the detected cancers, 98 were stage 0 or stage I.

Table 3 summarizes the outcomes data obtained for diagnostic examinations in our practice according to indication for examination. As has been reported previously, outcomes data for the probably benign category, the lumpectomy category, and the "other" (miscellaneous) category were generally similar [16]. The most obvious exception was the high rate of positive biopsy findings for lumpectomy cases (73% in our practice; reported in other studies to be approximately 50%) [19, 20]). For simplicity, we combined the three similar categories in producing Tables 4,5,6,7, which display combined diagnostic mammography outcomes data for many ranges of case mixes of indications for examination. We designed the data in these tables to be used by mammography practices that audit screening and diagnostic examinations separately but do not perform subset audits of their diagnostic mammography cases. Those practices that can determine (or estimate) their own case mixes of indications for examination can compare their observed diagnostic outcomes data with those presented in the appropriate table. Tables 4,5,6,7 should be used by practices for which the workup of abnormal screening examinations constitutes 10%, 20%, 30%, and 40% (respectively) of all diagnostic mammography examinations. Review of Tables 4,5,6,7 shows that combined diagnostic outcomes data are influenced primarily by the percentage of examinations performed to evaluate palpable masses.

Palpable Masses
As shown in Table 8, separate analysis of our diagnostic examinations performed for evaluation of palpable masses showed striking differences in clinical outcomes between this subset of examinations and the combination of all other diagnostic examinations, as well as the combination of all other diagnostic and screening examinations. Because it may be easier for a mammography practice to simply determine (or estimate) the percentage of its patients who have palpable masses, we produced Tables 9 and 10, which display estimates of combined outcomes data for many ranges of case mixes of palpable mass cases. We designed Table 9 to be used by mammography practices that audit screening and diagnostic examinations separately and Table 10 to be used by practices that do not segregate screening and diagnostic examinations during auditing.

As Table 9 indicates, a practice performing 5% of its diagnostic mammography examinations to evaluate palpable masses can expect to have different outcomes from a practice performing 70% of its examinations to evaluate palpable masses. For these two respective practices, the estimated rate of abnormal findings in 15% versus 20%, the estimated rate of positive biopsy results is 42% versus 56%, and the estimated cancer detection rate is 48 per 1,000 versus 100 per 1,000.

As Table 10 indicates, a practice performing 2% of its screening plus diagnostic mammography examinations to evaluate palpable masses can expect to have different outcomes from a practice performing 30% of its cases to evaluate palpable masses. For these two practices, respectively, the estimated rate of abnormal findings is 7% versus 12%, the estimated rate of positive biopsy findings is 40% versus 46%, and the estimated cancer detection rate is 15 per 1,000 versus 46 per 1,000.

Discussion

Top Abstract Introduction Materials and Methods Results Discussion References

 
A comprehensive medical audit of a mammography practice is a powerful tool to assess the ability to detect clinically occult, earlystage breast cancer. Such an audit serves primarily as a self-assessment device, revealing both successes and deficiencies in a given practice. Data from many audits of large screening mammography practices have been published, from both academic centers and community practices [3,4,5, 7,8,9,10,11,12,13]. However, reported data about diagnostic mammography outcomes are scarce [15, 16]. Nonetheless, as documented in these reports and reinforced by the data presented in our tables, audit outcomes data are different for screening versus diagnostic examinations. Therefore, the practice that performs audits without segregating screening and diagnostic mammography data can expect difficulty in analyzing the observed outcomes because of the confounding effects of combining such disparate data.

Unfortunately, many practices have neither the time nor the financial resources to perform comprehensive segregated audits. In this situation, analysis of combined data should be based on known differences between diagnostic and screening outcomes. By using the data reported in this article, which involve concurrently collected, consecutive screening and diagnostic mammography examinations, we derived outcomes data for a wide range of case mixes of screening-to-diagnostic examinations (Table 2), so that a practice that can determine (or estimate) its own case mix can then use our derived data as a benchmark with which to compare their observed outcomes.

Diagnostic mammography examinations are performed for a variety of problem-solving indications. Substantial differences in mammography outcomes are found when diagnostic examinations themselves are segregated by indication for examination [16]. Again, a mammography practice should expect its own diagnostic outcomes data to vary depending on its specific mix of indications for examinations. Therefore, we provide Tables 4,5,6,7 to be used by the diagnostic mammography practice that can determine (or estimate) its percentage of examinations done for each indication, so that its medical audit may be more meaningful.

Our results support data found in other smaller series of patients undergoing diagnostic mammography for palpable masses [21,22,23]. As shown in Table 8, the effect of palpable masses on audit outcomes data is so substantial that we suggest that a practice can simply derive expected outcome data if it can determine (or estimate) the percentage of examinations it performed for the evaluation of palpable masses. Tables 9 and 10 serve this purpose.

The data reported herein come from an academic practice in which most cases are interpreted by full-time breast imaging specialists. These data, therefore, may not be representative of the performance of general diagnostic radiologists in community practice. However, our series of concurrent screening and diagnostic mammography examinations is now the only published set of data with a sufficiently large number of cases to use for estimating outcomes. As other large consecutive series of comprehensively audited combinations of screening plus diagnostic cases are reported, especially if some come from community practices, a more meaningful data set can be used to derive the outcomes estimates displayed in the tables of this article.

References

Top Abstract Introduction Materials and Methods Results Discussion References

 

1. United States Food and Drug Administration. Quality mammography standards: final rule. 21 CFR 16 and 900. Washington, DC: United States Department of Health and Human Services, 1997
2. American College of Radiology. Breast imaging reporting and data system (BI-RADS), 3rd ed. Reston, VA: American College of Radiology, 1998
3. Sickles EA, Ominsky SH, Sollitto RA, Galvin HB, Monticciolo DL. Medical audit of a rapid-throughput mammography screening practice: methodology and results of 27,114 examinations. Radiology 1990;175:323 -327[Abstract/Free Full Text]
4. Sickles EA. Auditing your practice. In: Kopans DB, Mendelson EB, eds. Syllabus: a categorical course in breast imaging. Oak Brook, IL: Radiological Society of North America, 1995: 81-91
5. Sickles EA. Quality assurance: how to audit your own mammography practice. Radiol Clin North Am 1992;30:265 -275[Medline]
6. Linver MN, Osuch JR, Brenner RJ, Smith RA. The mammography audit: a primer for the Mammography Quality Standards Act (MQSA). AJR 1995;165:19 -25[Abstract/Free Full Text]
7. Linver MN, Paster SB, Rosenberg RD, et al. Improvement in mammography interpretation skills in a community radiology practice after dedicated teaching courses: 2-year medical audit of 38,633 cases. Radiology 1992;184:39 -43[Abstract/Free Full Text]
8. Rosenberg RD, Lando JF, Hunt WC, et al. The New Mexico Mammography Project: screening mammography performance in Albuquerque, New Mexico, 1991 to 1993. Cancer 1996;78:1731 -1739[Medline]
9. Tabár L, Fagerberg G, Duffy SW, Day NE, Gad A, Gröntoft O. Update of the Swedish two-county program of mammographic screening for breast cancer. Radiol Clin North Am 1992;30:187 -210[Medline]
10. Lynde JL. Low-cost screening mammography: results of 21,141 consecutive examinations in a community program. South Med J 1993;86:338 -343[Medline]
11. Thurfjell E. Population-based mammography screening in clinical practice: results from the prevalence round in Uppsala county. Acta Radiol 1994;35:487 -491[Medline]
12. Thurfjell EL, Lindgren JA. Breast cancer survival rates with mammographic screening: similar favorable rates for women younger and those older than 50 years. Radiology 1996;201:421 -426[Abstract/Free Full Text]
13. Burhenne LJ, Burhenne HJ, Kan L. Quality-oriented mass mammography screening. Radiology 1995;194:185 -188[Abstract/Free Full Text]
14. Bassett LW, Hendrick RE, Bassford TL, et al. Quality determinants of mammography: clinical practice guideline no. 13. Rockville, MD: Agency for Health Care Policy and Research, Public Health Service, United States Department of Heath and Human Services, 1994. AHCPR publication 95-0632
15. Robertson CL. A private breast imaging practice: medical audit of 25,788 screening and 1,077 diagnostic examinations. Radiology 1993;187:75 -79[Abstract/Free Full Text]
16. Dee KE, Sickles EA. Medical audit of diagnostic mammography examinations: comparison with screening outcomes obtained concurrently. AJR 2001;176:729 -733[Abstract/Free Full Text]
17. Sickles EA. The usefulness of computers in managing the operation of a mammography screening practice. AJR 1990;155:755 -761[Abstract/Free Full Text]
18. American Joint Committee on Cancer. Manual for staging of cancer, 5th ed. Philadelphia: Lippincott, 1997
19. Stomper PC, Recht A, Berenberg AL, Jochelson MS, Harris JR. Mammographic detection of recurrent cancer in the irradiated breast. AJR 1987;148:39 -43[Abstract/Free Full Text]
20. Orel SG, Troupin RH, Patterson EA, Fowble BL. Breast cancer recurrence after lumpectomy and irradiation: role of mammography in detection. Radiology 1992;183:201 -206[Abstract/Free Full Text]
21. Bassett LW, Liu TH, Giuliano AE, Gold RH. The prevalence of carcinoma in palpable vs impalpable, mammographically detected lesions. AJR 1991;157:21 -24[Abstract/Free Full Text]
22. Rosen EL, Sickles E, Keating D. Ability of mammography to reveal nonpalpable breast cancer in women with palpable breast masses. AJR 1999;172:309 -312[Abstract/Free Full Text]
23. Roubidoux MA, Helvie MA, Lai NE, Paramagul C. Bilateral breast cancer: early detection with mammography. Radiology 1995;196:427 -431[Abstract/Free Full Text]

CiteULike

Complore

Connotea

Del.icio.us

Digg

Reddit

Technorati

This article has been cited by other articles:

				E. A. Sickles, D. L. Miglioretti, R. Ballard-Barbash, B. M. Geller, J. W. T. Leung, R. D. Rosenberg, R. Smith-Bindman, and B. C. Yankaskas Performance Benchmarks for Diagnostic Mammography Radiology, June 1, 2005; 235(3): 775 - 790. [Abstract] [Full Text] [PDF]

				R. Smith-Bindman, P. W. Chu, D. L. Miglioretti, E. A. Sickles, R. Blanks, R. Ballard-Barbash, J. K. Bobo, N. C. Lee, M. G. Wallis, J. Patnick, et al. Comparison of Screening Mammography in the United States and the United Kingdom JAMA, October 22, 2003; 290(16): 2129 - 2137. [Abstract] [Full Text] [PDF]

				W. A. Berg Rationale for a Trial of Screening Breast Ultrasound: American College of Radiology Imaging Network (ACRIN) 6666 Am. J. Roentgenol., May 1, 2003; 180(5): 1225 - 1228. [Full Text] [PDF]

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Sohlich, R. E. Articles by Dee, K. E. Search for Related Content PubMed PubMed Citation Articles by Sohlich, R. E. Articles by Dee, K. E. Social Bookmarking                      What's this?