In the evaluation of suspected ureteral obstruction, an international consensus panel recommended baseline imaging with technetium-99m mercaptoacetyltriglycine (^99mTc MAG3) followed by furosemide administration and 15 minutes of additional imaging [1]. Some investigators have recommended strategies to eliminate the need for furo-semide in selected patients [2]. The authors of the consensus report did note that baseline imaging might be sufficient to exclude obstruction in experienced hands, but this option was not their general recommendation [1]. Nevertheless, if the baseline examination can exclude obstruction, the patient can be spared the administration of furosemide and additional imaging time, and there will also be a cost saving because of the reduction of camera, computer, and technologist time required to complete the furosemide component of the study and physician time required to interpret it.

Despite these advantages, it may be challenging for a general radiologist to acquire the experience and expertise in diuresis renography to determine with confidence whether a kidney is obstructed or when furosemide is not required. Radiologists are required to master an ever-expanding knowledge base while the hours available to master this knowledge base and apply it to specific tasks are steadily shrinking because of pressure to increase the number of studies each radiologist interprets. The convergence of an expanding knowledge base and escalating time constraints inevitably increases the risk of physician error. This is particularly true of low-volume studies such as diuresis renography in which radiologists may have had limited training and experience. In fact, a large percentage of the estimated 590,000 renal imaging examinations performed annually in the United States are interpreted at sites that perform fewer than three studies per week [3]. For these reasons, it is particularly important to develop and implement decision support tools to assist physicians in interpreting low-volume studies so that such studies can be interpreted at a faster rate and at a higher level of expertise.

The digital and dynamic nature of ^99mTc MAG3 renal scanning data has made it possible to quantify a variety of functional renal parameters to produce an expanded knowledge base. Two independent renal decision support systems (RENEX and CARTAN) are being developed to use this expanded knowledge base to determine the need for furosemide and to detect renal obstruction in patients referred for diuresis renography [4, 5]. RENEX (renal expert) uses a set of rules obtained from human experts to analyze the quantitative parameters obtained from MAG3 scintigraphy, whereas CARTAN (classification and regression tree analysis) is a statistical method that grows and prunes a decision tree on the basis of an analysis of these quantitative parameters in a training data set [6].

As an intermediate step to developing a decision support system to detect obstruction, this prospective study was performed to compare the decisions regarding the need for furosemide made by a heuristic approach (RENEX) and an analytic approach (CARTAN) with the need for furosemide determined in clinical practice and by expert reviewers.

This study was performed under the purview and with the approval of the internal review board at Emory University School of Medicine. Data collection and database use were compliant with the terms of the Health Insurance Portability and Accountability Act (HIPPA).

The baseline plus furosemide protocol recommended in the 1996 consensus report on diuresis renography appears to be in common use [1]. On a show of hands or keypad response, 50%, 90%, 85%, and 80% of the participants attending the 2005 Educational Symposia Nuclear Medicine course, the 2005 Emory course in Nuclear Medicine, the 2005 Northeast Regional Meeting of the SNM, and a 2005 RSNA educational course, respectively, used the standard protocol as either two separate acquisitions or a continuous acquisition with furosemide administered after approximately 20 minutes.

Regarding the need for furosemide, RENEX agreed with the clinical decision to give furosemide in 97% of patients (68/70), whereas CARTAN agreed in 90% of patients (63/70), p = 0.03 (Table 1). Similarly, RENEX agreed with the experts' decision to give furosemide in 98% of patients (65/66), whereas CARTAN agreed with 89% (59/66), p = 0.01 (Table 1). When the data were analyzed by kidney rather than by patient, RENEX agreed with the experts' decision to give furosemide in 94% of kidneys (84/89) compared with 82% (73/89) for CARTAN, p = 0.002. Sample studies are illustrated in Figures 2A, 2B and 3A, 3B. The experts' decision agreed with the clinical decision to give furosemide in 94% of patients (66/70).

There was no significant difference between RENEX and CARTAN when the clinicians and experts deemed that furosemide was not necessary, p = 0.18 and 0.32, respectively (Table 2). However, when the data were analyzed by individual kidneys (Table 2), CARTAN agreed with the experts' decision in 78% of kidneys (87/111), whereas RENEX agreed in 69% of kidneys (77/111), p = 0.008. The experts' decision agreed with the clinical decision to withhold furosemide in 100% (32/32) of patients.

Overall, RENEX agreed with the clinical decision in 89% (91/102) of patients, with the experts' decision in 87% (89/102) of patients, and with the experts' decision in 81% (161/200) of kidneys. CARTAN agreed with the clinical decision in 87% (89/102) of patients, with the experts' decision in 83% (85/102) of patients, and with the experts' decision in 80% (160/200) of kidneys. The experts' decision agreed with the clinical decision to give or withhold furosemide in 96% (98/102) of patients.

The performance of RENEX and CAR-TAN in the right kidney was compared with the performance in the left kidney. RENEX agreed with experts' decision to administer furosemide in 93% (40/43) of left kidneys and 96% (44/46) of right kidneys, p = not statistically significant (Table 3). Comparable results were obtained for CARTAN (Table 3). Similarly, there was no significant difference between the right and left kidneys for RENEX and CARTAN when the results were analyzed based on the experts' decisions to withhold furosemide (Table 4).

Over the past several years, artificial intelligence methods have been investigated as a way to develop tools to improve diagnostic performance. Examples from nuclear medicine include neural networks [23-25] and case-based reasoning techniques [26] to provide computer-assisted diagnosis of planar and SPECT myocardial perfusion studies. In knowledge-based expert systems, a knowledge base of heuristic rules is obtained from human experts capturing how they make their interpretations. These rules are usually expressed in the form of IF A THEN B expressions.

One major advantage of a knowledge-based expert system is that the system does not require the same large numbers of studies as are required to develop neural networks or case-based reasoning approaches. A second advantage, especially from a learning perspective, is that it is possible to query a knowledge-based system to learn the rules that led to a specific conclusion. For example, RENEX disagreed with the experts regarding the need for furosemide in the right kidney (Fig. 3A, 3B), but it is possible to query RENEX to determine the reasons for its decision. The primary reason RENEX concluded furosemide was needed was that the half-time (T_½) was abnormal (20.8 minutes); RENEX recognized that there was moderate evidence that furosemide was not needed because the postvoid to maximum ratio was 0.2 (upper limits of the normal value of 0.16) [15], but RENEX gave greater weight to the abnormal T_½. The experts were not queried but clearly gave greater weight to the images and the low postvoid to maximum ratio. RENEX can be improved by comparing results and “reasoning” with expert decisions and adding new rules or adjusting the weighting factors. In this case, RENEX should be tested giving greater weight to the postvoid to maximum count ratio than the T_½.

We chose to implement a purely statistical method (CARTAN) for the second decision support approach because the combined approach of classification and regression tree (CART) analysis with bootstrapping could handle a relatively large number of variables and arrive at a prediction algorithm with a relatively small training set.

CART analysis, but not bagging, has been used to help distinguish between healthy and diseased kidneys using MR data that include measurements of renal arterial blood flow and parenchymal perfusion [27]. To our knowledge, this is the first time CART analysis and bagging have been applied to a problem in nuclear medicine. CART agreed with the expert decisions illustrated in Figure 3A, 3B, but CART cannot provide the rationale for these decisions. However, CART can use a large sample of test patients to determine the parameters and parameter cutoff values that provide the best separation in distinguishing when furosemide is needed or not needed or whether a kidney is obstructed. These parameters and cutoff values can be used to guide clinicians and RENEX in determining the need for furosemide and the presence or absence of obstruction.

Our long-term goal is to develop decision support systems to determine if a kidney is obstructed. The dual statistical and knowledge-based approaches have provided complementary information and helped target features that can be transferred or emphasized to improve the overall performance of both systems. At present, data are insufficient to determine if one approach is inherently superior to the other. Analysis of the baseline study to determine the need for furosemide may be useful for the 50-90% of institutions that use a baseline plus furosemide protocol. These decision support systems have the potential to serve as a second opinion to support a clinician's preliminary decision regarding the need for furosemide. This study has also provided important data and direction for developing decision support systems to diagnose obstruction.

The focus of this research effort is to develop statistical and knowledge-based decision support systems that will have diagnostic performance in diuresis renography comparable to that of experts. This is not a trivial task; however, it could be argued that a better goal would be to develop decision support systems that use the clinical out-come as the gold standard rather than expert reviewers. This is an attractive goal, but it is confounded by the fact that an obstructed or nonobstructed scan interpretation has a major impact on the clinical outcome. Consequently, this gold standard becomes biased.

Additional problems of data analysis can be illustrated by a patient who had a pyeloplasty to relieve obstruction 6-24 months after a diuresis renography scan was interpreted as no obstruction. In this example, did the scan miss the obstruction, was the study interpreted incorrectly, or did the patient become obstructed 6-24 months after the scan? Using patient outcome as a gold standard can be an important goal, but interpretation of the results is not straightforward. Moreover, regardless of the type of study, clinical practice would be improved if radiologists can provide an interpretation equivalent to that of expert reviewers.

The study has several limitations. The training set was relatively small; this is probably more of a disadvantage for a statistical system such as CARTAN than for a knowledge-based system, but both of these systems could potentially be improved by increasing the size of the training set. This study addressed the diuresis renography protocol recommended by the international consensus report in which baseline data are obtained, followed by the administration of furosemide and an additional period of imaging [1]. There are other protocols in which furosemide is given 15 minutes before the radiopharmaceutical, at the same time as the radiopharmaceutical, or 5 to 10 minutes later [1, 28, 29]. Obviously, the systems we describe do not apply to these protocols Finally, QuantEM 2.0 cannot detect and correct for patient motion, and at this time the software cannot distinguish between diffuse retention with slow washout due to impaired function and focal pelvic retention with slow washout due to possible obstruction. Algorithms to detect and correct for motion and to distinguish between diffuse retention in a kidney and retention in a dilated renal collecting system need to be designed, implemented, and tested.

In conclusion, RENEX performed better than CARTAN when clinicians and experts determined that furosemide was necessary to evaluate obstruction. When furosemide was not required, CARTAN performed better than RENEX for cases in which the results were analyzed by individual kidney; there was no difference when the results were analyzed by patient. Use of RENEX or CARTAN as decision support tools in the baseline plus furosemide protocol has the potential to help avoid unnecessary imaging; minimize patient inconvenience; and reduce costs by saving technologist, camera, computer, and physician time. Notably, the results obtained from this study will assist in the more complex task of developing decision support systems for diuresis renography to evaluate obstruction.

We thank Russell D. Folks, Angela Hunsche, and Meghna Krishnan for their assistance in collecting and archiving the data used in this study.

Address correspondence to A. Taylor ([email protected]).

Supported by grant ROI LMN07595 from the Library of Medicine.