We read with interest the article by Kovacs and colleagues [1] in the March 2015 issue of AJR in which the authors evaluated the effect of the JPEG 2000 80:1 lossy compression algorithm on digital mammograms. Kovacs et al. also proposed that the results of their study warrant a change in the current U.S. Food and Drug Administration (FDA) policy which does not permit the use of lossy compression of digital mammograms for primary interpretation and image retention. We would like to address some broad issues concerning compression and image quality in digital mammography and some specific concerns about this study.

The Mammography Quality Standards Act (MQSA) is intended to ensure that mammography services meet specified quality standards. Among other requirements, MQSA regulations require that a mammography facility maintain the original mammograms for a specified period of time and, on the patient's request, transfer the mammograms to the patient or a designated recipient. The FDA currently interprets MQSA as prohibiting lossy compression of digital mammograms for primary image interpretation, image retention, or transfer to the patient or her designated recipient. These requirements are intended to preserve the quality of the clinical images because inappropriate use of lossy compression can lead to the loss of pertinent information in the image and thereby affect various clinical tasks.

Many factors must be considered when comparing lossy-compressed medical images to either uncompressed or lossless-compressed medical images. In addition to lesion detection, there are other clinical tasks, such as lesion measurement, volume estimation, and lesion characterization, on which the effects of lossy compression must be investigated. Furthermore, automated algorithms for computer-aided detection and for lesion segmentation and volume estimation are being used more widely, and it is well known that the performance of these systems can be affected by data compression [2].

Another important issue is the choice of the metric used to describe the extent of compression. Kovacs et al. [1] characterized the compression algorithm used in their study by its compression ratio. However, studies indicate that the quality of a compressed medical image at a given compression ratio depends on multiple factors, including the imaging modality, imaging parameters, tissue characteristics, lesion characteristics, and compression algorithm [3, 4]. This is not surprising because the rate at which an image can be compressed without exhibiting artifacts or affecting diagnostic accuracy is dependent on the content of the individual image. The compression ratio merely indicates the relative reduction in file size of an image, and it would be incorrect to infer the quality of an individual image on the basis of the compression ratio alone. When using compression ratio as a measure in a study, the results can be generalized to the entire patient population only if sufficient sampling and analyses of subpopulations of patients under different imaging conditions have been performed. This study used a limited sample size and provided no analysis of subpopulations such as patients with dense breast tissue, extensive coarse calcifications, or extensive noncalcified lesions. Therefore, the study results cannot be generalized to the entire mammography patient population.

Because of the high costs of exploring all parameters that can affect image quality, subjective studies of image compression typically assess only a small number of images that cannot represent the full population of patient images. This study design leads to often conflicting conclusions about acceptable rates of compression, as was also noted by Kovacs et al. [1]. Objective image quality metrics can explore the parameter space more efficiently and generate a more reliable estimate of the effect of a compression algorithm on the quality of an individual image. The use of a metric that captures quality rather than compression ratio provides greater assurance that all patient images will be processed appropriately even when using lossy compression. If quality metrics (e.g., metrics described in [5]) are validated for future studies of medical image compression, they could complement or replace subjective evaluation and lead to significant cost savings and safety assurance.

We also have some concerns about the statistical analysis in this study including the formulation of the hypothesis, the choice of assessment paradigm, and the paucity of CIs. First, failure to reject the null hypothesis does not imply that the null hypothesis is correct. Moreover, the probability of failing to reject a false null hypothesis (type II error) is not controlled in conventional hypothesis testing. The study should therefore be formulated as a noninferiority test in which the null hypothesis is inferiority and the alternative hypothesis is noninferiority [6, 7]. Also, jackknife alternative free-response ROC (JAFROC) analysis is inappropriate for the data that were collected. The JAFROC figure of merit is designed for free-response data (i.e., the reader marks all suspicious locations on each case and scores each mark [8]). However, in this study, only the most suspicious location was marked. More appropriate assessment paradigms include ROC [9] and localization ROC (LROC) [10]. Finally, the authors reported only a single CI for the JAFROC figure of merit. CIs for the other figures of merit (lesion localization fraction, true-positive fraction, false-positive fraction) should also have been reported.

We agree that digital mammography and breast tomosynthesis have led to an increase in the volume of data generated by the practice of mammography. However, off-site “cloud” data storage and sharing solutions now exist that permit both physicians and patients to rapidly access, view, and download full digital mammograms and full tomosynthesis images without the need for lossy compression and with appropriate privacy safeguards. In time, these solutions will likely become even more widely available and less expensive.

A primary goal of MQSA remains the preservation of clinical image quality. Before considering the acceptance of lossy data compression for mammographic interpretation and retention, we recommend that further studies be performed, using appropriate quality metrics, study designs, and statistical analyses, to determine the impact of compression on clinical images.