November 2021, VOLUME 217

Recommend & Share

November 2021, Volume 217, Number 5

Genitourinary Imaging

Original Research

Assessment of Renal Cell Carcinoma by Texture Analysis in Clinical Practice: A Six-Site, Six-Platform Analysis of Reliability

+ Affiliations:
1Department of Radiology, New York University Langone Medical Center, 660 First Ave, New York, NY 10016

2Department of Radiology, University of Michigan Health Systems, Ann Arbor, MI

3Department of Radiology, University of Alabama at Birmingham, Birmingham, AL

4Department of Medical Imaging, The Ottawa Hospital, The University of Ottawa, Ottawa, ON, Canada

5Department of Radiology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA

6Department of Diagnostic Radiology, The University of Texas M. D. Anderson Cancer Center, Houston, TX

Citation: American Journal of Roentgenology. 2021;217: 1132-1140. 10.2214/AJR.21.25456


BACKGROUND. Multiple commercial and open-source software applications are available for texture analysis. Nonstandard techniques can cause undesirable variability that impedes result reproducibility and limits clinical utility.

OBJECTIVE. The purpose of this study is to measure agreement of texture metrics extracted by six software packages.

METHODS. This retrospective study included 40 renal cell carcinomas with contrast-enhanced CT from The Cancer Genome Atlas and Imaging Archive. Images were analyzed by seven readers at six sites. Each reader used one of six software packages to extract commonly studied texture features. Inter- and intrareader agreement for segmentation was assessed with intraclass correlation coefficients (ICCs). First-order (available in six packages) and second-order (available in three packages) texture features were compared between software pairs using Pearson correlation.

RESULTS. Inter- and intrareader agreement was excellent (ICC, 0.93–1). First-order feature correlations were strong (r ≥ 0.8, p < .001) between 75% (21/28) of software pairs for mean intensity and SD, 48% (10/21) for entropy, 29% (8/28) for skewness, and 25% (7/28) for kurtosis. Of 15 second-order features, only cooccurrence matrix correlation, gray-level nonuniformity, and run-length nonuniformity showed strong correlation between software packages (r = 0.90–1, p < .001).

CONCLUSION. Variability in first- and second-order texture features was common across software configurations and produced inconsistent results. Standardized algorithms and reporting methods are needed before texture data can be reliably used for clinical applications.

CLINICAL IMPACT. It is important to be aware of variability related to texture software processing and configuration when reporting and comparing outputs.

Keywords: CT, radiomics, renal cell carcinoma, texture analysis

Based on a presentation at the Society of Abdominal Radiology 2019 annual meeting, Orlando, FL.

Previous section
1. Rosenkrantz AB, Mendiratta-Lala M, Bartholmai BJ, et al. Clinical utility of quantitative imaging. Acad Radiol 2015; 22:33–49 [Google Scholar]
2. Raman SP, Chen Y, Schroeder JL, Huang P, Fishman EK. CT texture analysis of renal masses: pilot study using random forest classification for prediction of pathology. Acad Radiol 2014; 21:1587–1596 [Google Scholar]
3. Erdim C, Yardimci AH, Bektas CT, et al. Prediction of benign and malignant solid renal masses: machine learning-based CT texture analysis. Acad Radiol 2020; 27:1422–1429 [Google Scholar]
4. Feng Z, Rong P, Cao P, et al. Machine learning-based quantitative texture analysis of CT images of small renal masses: differentiation of angiomyolipoma without visible fat from renal cell carcinoma. Eur Radiol 2018; 28:1625–1633 [Google Scholar]
5. Hodgdon T, McInnes MD, Schieda N, Flood TA, Lamb L, Thornhill RE. Can quantitative CT texture analysis be used to differentiate fat-poor renal angiomyolipoma from renal cell carcinoma on unenhanced CT images? Radiology 2015; 276:787–796 [Google Scholar]
6. Deng Y, Soule E, Samuel A, et al. CT texture analysis in the differentiation of major renal cell carcinoma subtypes and correlation with Fuhrman grade. Eur Radiol 2019; 29:6922–6929 [Google Scholar]
7. Ding J, Xing Z, Jiang Z, et al. CT-based radiomic model predicts high grade of clear cell renal cell carcinoma. Eur J Radiol 2018; 103:51–56 [Google Scholar]
8. Kocak B, Durmaz ES, Ates E, Kaya OK, Kilickesmez O. Unenhanced CT texture analysis of clear cell renal cell carcinomas: a machine learning-based study for predicting histopathologic nuclear grade. AJR 2019; 212:[web] W132–W139 [Abstract] [Google Scholar]
9. Feng Z, Shen Q, Li Y, Hu Z. CT texture analysis: a potential tool for predicting the Fuhrman grade of clear-cell renal carcinoma. Cancer Imaging 2019; 19:6 [Google Scholar]
10. Kocak B, Durmaz ES, Ates E, Ulusan MB. Radiogenomics in clear cell renal cell carcinoma: machine learning-based high-dimensional quantitative CT texture analysis in predicting PBRM1 mutation status. AJR 2019; 212:[web]W55–W63 [Abstract] [Google Scholar]
11. Goh V, Ganeshan B, Nathan P, Juttla JK, Vinayan A, Miles KA. Assessment of response to tyrosine kinase inhibitors in metastatic renal cell cancer: CT texture as a predictive biomarker. Radiology 2011; 261:165–171 [Google Scholar]
12. Lubner MG, Smith AD, Sandrasegaran K, Sahani DV, Pickhardt PJ. CT texture analysis: definitions, applications, biologic correlates, and challenges. RadioGraphics 2017; 37:1483–1503 [Google Scholar]
13. Traverso A, Wee L, Dekker A, Gillies R. Repeatability and reproducibility of radiomic features: a systematic review. Int J Radiat Oncol Biol Phys 2018; 102:1143–1158 [Google Scholar]
14. Berenguer R, Pastor-Juan MDR, Canales-Vázquez J, et al. Radiomics of CT features may be nonreproducible and redundant: influence of CT acquisition parameters. Radiology 2018; 288:407–415 [Google Scholar]
15. Varghese BA, Cen SY, Hwang DH, Duddalwar VA. Texture analysis of imaging: what radiologists need to know. AJR 2019; 212:520–528 [Abstract] [Google Scholar]
16. Gillies RJ, Kinahan PE, Hricak H. Radiomics: images are more than pictures, they are data. Radiology 2016; 278:563–577 [Google Scholar]
17. Heye T, Davenport MS, Horvath JJ, et al. Reproducibility of dynamic contrast-enhanced MR imaging. Part I. Perfusion characteristics in the female pelvis by using multiple computer-aided diagnosis perfusion analysis solutions. Radiology 2013; 266:801–811 [Google Scholar]
18. Clark K, Vendt B, Smith K, et al. The Cancer Imaging Archive (TCIA): maintaining and operating a public information repository. J Digit Imaging 2013; 26:1045–1057 [Google Scholar]
19. Kirby J, Jarosz Q. TCGA-KIRC. The Cancer Imaging Archive website. Published 2016. Accessed December 6, 2017 [Google Scholar]
20. Schieda N, Krishna S, McInnes MDF, et al. Utility of MRI to differentiate clear cell renal cell carcinoma adrenal metastases from adrenal adenomas. AJR 2017; 209:[web]W152–W159 [Abstract] [Google Scholar]
21. Fedorov A, Beichel R, Kalpathy-Cramer J, et al. 3D Slicer as an image computing platform for the Quantitative Imaging Network. Magn Reson Imaging 2012; 30:1323–1341 [Google Scholar]
22. van Griethuysen JJM, Fedorov A, Parmar C, et al. Computational radiomics system to decode the radiographic phenotype. Cancer Res 2017; 77:e104–e107 [Google Scholar]
23. Schieda N, Lim RS, Krishna S, McInnes MDF, Flood TA, Thornhill RE. Diagnostic accuracy of unenhanced CT analysis to differentiate low-grade from high-grade chromophobe renal cell carcinoma. AJR 2018; 210:1079–1087 [Abstract] [Google Scholar]
24. Zou KH, Tuncali K, Silverman SG. Correlation and simple linear regression. Radiology 2003; 227:617–622 [Google Scholar]
25. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016; 15:155–163 [Google Scholar]
26. Pyradiomics website. Radiomic features. Accessed March 31, 2021 [Google Scholar]
27. Foy JJ, Robinson KR, Li H, Giger ML, Al-Hallaq H, Armato SG 3rd. Variation in algorithm implementation across radiomics software. J Med Imaging (Bellingham) 2018; 5:044505 [Google Scholar]
28. Liang ZG, Tan HQ, Zhang F, et al. Comparison of radiomics tools for image analyses and clinical prediction in nasopharyngeal carcinoma. Br J Radiol 2019; 92:20190271 [Google Scholar]
29. Yip SS, Aerts HJ. Applications and limitations of radiomics. Phys Med Biol 2016; 61:R150–R166 [Google Scholar]
30. Doumou G, Siddique M, Tsoumpas C, Goh V, Cook GJ. The precision of textural analysis in (18)F-FDG-PET scans of oesophageal cancer. Eur Radiol 2015; 25:2805–2812 [Google Scholar]
31. Desseroit MC, Tixier F, Weber WA, et al. Reliability of PET/CT shape and heterogeneity features in functional and morphologic components of non-small cell lung cancer tumors: a repeatability analysis in a prospective multicenter cohort. J Nucl Med 2017; 58:406–411 [Google Scholar]
32. Schwier M, van Griethuysen J, Vangel MG, et al. Repeatability of multipara-metric prostate MRI radiomics features. Sci Rep 2019; 9:9441 [Google Scholar]
33. Zwanenburg A, Vallières M, Abdalah MA, et al. The image biomarker standardization initiative: standardized quantitative radiomics for high-throughput image-based phenotyping. Radiology 2020; 295:328–338 [Google Scholar]
34. Kocak B, Durmaz ES, Kaya OK, Ates E, Kilickesmez O. Reliability of single-slice-based 2D CT texture analysis of renal masses: influence of intra- and interobserver manual segmentation variability on radiomic feature reproducibility. AJR 2019; 213:377–383 [Abstract] [Google Scholar]
Address correspondence to A. M. Doshi ().

H. Rusinek is a codeveloper of FireVoxel. The remaining authors declare that they have no disclosures relevant to the subject matter of this article.

Recommended Articles

Assessment of Renal Cell Carcinoma by Texture Analysis in Clinical Practice: A Six-Site, Six-Platform Analysis of Reliability

No Access, , , , ,
American Journal of Roentgenology. 2021;216:1549-1557. 10.2214/AJR.20.22823
Abstract | Full Text | PDF (940 KB) | PDF Plus (973 KB) | Supplemental Material 
Full Access, , , , , , , , , , , , , , , , , , , , , , , ,
American Journal of Roentgenology. 2021;217:859-869. 10.2214/AJR.21.25499
Abstract | Full Text | PDF (1016 KB) | PDF Plus (1046 KB) 
Full Access, , , , , , , , , , , ,
American Journal of Roentgenology. 2021;217:1093-1102. 10.2214/AJR.21.25640
Abstract | Full Text | PDF (841 KB) | PDF Plus (817 KB) 
Full Access, ,
American Journal of Roentgenology. 2021;217:898-906. 10.2214/AJR.21.25545
Abstract | Full Text | PDF (898 KB) | PDF Plus (923 KB) 
No Access, , , , ,
American Journal of Roentgenology. 2021;217:919-920. 10.2214/AJR.21.25732
Abstract | Full Text | PDF (533 KB) | PDF Plus (528 KB) | Supplemental Material 
No Access, , , ,
American Journal of Roentgenology. 2021;217:541-555. 10.2214/AJR.21.25493
Abstract | Full Text | PDF (1437 KB) | PDF Plus (1295 KB)