The goal of an image-based classification system is to characterize adnexal lesions as benign or possibly malignant. In this study, we used two methods—the IOTA simple rules and the criteria described in the SRU consensus guidelines—to classify adnexal cystic lesions in a general population of women presenting to academic radiology departments. All 20 malignancies were classified according to the SRU guidelines into indeterminate or malignant categories. One early-stage endometrioid carcinoma was classified as benign according to the IOTA simple rules. The SRU guidelines were more sensitive (100%), and the IOTA simple rules were more specific (96.5%) for malignancy. Our results are similar to previously reported sensitivities and specificities for image-based adnexal lesion classification systems [4, 5, 22–24]. Both the IOTA simple rules and the SRU guidelines had high NPVs (99.8–100%) for excluding malignancy, and low PPVs (14.9–29%) for malignancy.

From ROC analysis, the performance of the IOTA simple rules was slightly more accurate than that of the SRU guidelines for the diagnosis of malignancy (p < 0.003) because fewer benign lesions were misclassified into the indeterminate and malignant categories with the IOTA simple rules. The clinical relevance of this small difference in clinical practice is uncertain. Using these image-based criteria in clinical practice depends on the goals of the practice. If ultrasound is used predominantly as a screening tool to exclude cancer, then the higher sensitivity and NPV of the SRU guidelines would tip the scales toward this approach. On the other hand, if the primary goal of ultrasound is lesion characterization and treatment planning, the higher specificity and PPV of the IOTA simple rules would favor their use.

In our study, both indeterminate and malignant categories harbored malignant lesions; approximately one-third of pathologically proven malignancies were categorized into the indeterminate group by use of SRU guidelines and IOTA simple rules. The prevalence of malignancy in the indeterminate category was 4.8% (7/145) for SRU to 10.7% (7/65) for IOTA and in the malignant category was 18.1% (13/72) for SRU to 34.3% (12/35) for IOTA. Previous investigators have reported the prevalence of malignancy within ultrasound indeterminate lesions as ranging between 5% and 40%, depending on the inclusion criteria and reader experience [12, 25, 26]. The SRU guidelines call for referring patients with lesions categorized as indeterminate for either imaging follow-up or surgical consultation [11]. Referring for imaging surveillance risks losing the window for catching early-stage cancer, whereas referring for surgery risks overtreatment. Secondary examinations such as MRI can help avoid unnecessary surgical referrals [21, 27]. A 2017 study [21] showed that use of MRI can reduce the number of surgical evaluations for benign cysts by 89%. This is not surprising given that MRI has a high NPV (100%) when there is a lack of solid tissue enhancement [28].

In our study, a lesion categorized as benign carried little to no risk of malignancy (NPV, 99.8–100%). However, 5–10% of the lesions characterized as benign according to imaging criteria were benign neoplasms, such as dermoid cysts and cystadenomas, and one early-stage ovarian carcinoma was classified as benign with the IOTA simple rules. Our study was not designed to determine which lesions classified as benign on the basis of image-based criteria require follow-up. However, the findings of our study underscore the need for additional research to determine the role of follow-up of benign-appearing lesions, some of which may ultimately be found to be benign neoplasms or very early ovarian cancer [11]. Follow-up imaging was helpful in the case of the patient in Figure 4, who was undergoing follow-up because the original clinical reader recommended additional follow-up because of a single small (4 mm) avascular papillary projection. This lesion did exhibit ultrasound evidence of continued growth, eventually prompting removal of this stage I endometrioid cancer.

With both methods, a malignant image-based categorization conveyed some risk of malignancy (PPV, 14.9–29.0%). However, the PPV in our study cohort is lower than that of the cohort studied by the IOTA group [12]; a PPV of 87% can be extrapolated from their results. This disparity likely arises from differences in the populations analyzed, technical expertise of the person performing the ultrasound examination (sonographer versus physician), and cancer incidence. In our study, we evaluated lesions in a population of women presenting to academic radiology departments for a variety of indications who underwent scanning performed by sonographic technologists, and only one-third of the women underwent surgery. The 2010 IOTA study [12] evaluated a population referred to a gynecologist for a suspected adnexal mass, the ultrasound was performed by trained gynecologists or radiologists with more than 5 years' experience, and all women underwent surgery. The prevalence of malignancy in our population was 2.6% versus the 27.9% in the IOTA study; the higher prevalence of cancer increases the PPV.

A 2018 study [29] also showed lower PPV with the IOTA simple rules in an outpatient setting. The authors of that study noted that in the outpatient setting, the IOTA simple rules retained high sensitivity, but the lower specificity indicated the need for a secondary test to help minimize false-positive findings. These differences in performance of the IOTA simple rules between studies underscore the potential variable performance of image-based classification systems related to patient population, practice setting, and prevalence of cancer [22, 29].

The major limitations of our study were that our population was primarily premenopausal with a high number of benign lesions and low prevalence of malignancy and that our retrospective approach did not allow control over ultrasound image quality and technical parameters. These image-based classification methods may perform differently in different populations and practice conditions. In addition, inherent in the retrospective approach is potential sampling bias in this population because a portion of the patients were lost to follow-up (38%). It is not known whether the lesions in these patients were ultimately found to be benign or malignant. Finally, a three-category system (malignant, benign, indeterminate) for the SRU guidelines had to be applied post hoc for the comparative analysis between the two methods. This required making several inferences to create the numeric categorization, which might not completely reflect the thoughts of the SRU expert panel. There also was a limitation in regard to differences between the SRU guidelines and IOTA simple rules in years of training and level of familiarity, which may have affected the results.

Image-based classification systems are sensitive for the identification of ovarian malignancy. However, the PPV is low in a general population of predominately premenopausal women presenting to radiology departments, and the indeterminate classification harbored one-third of the total malignancies. Exploration of varied clinical settings and inclusion of secondary tests may help to refine these systems and achieve the balanced goals of avoiding inappropriate surgery for benign cystic lesions and expediting the care of women with potentially malignant cystic tumors.

We thank members of the Society of Abdominal Radiology Uterine and Ovarian Cancer Disease-Focused Panel for performing research for this study.