OBJECTIVE. The purpose of this study was to evaluate the reproducibility of measurements of minimal distance from an invasive tumor to the anticipated circumferential resection margin in prediction of depth of extramural tumor spread in patients with rectal cancer.
MATERIALS AND METHODS. Images from 168 consecutive pelvic MRI examinations of patients with rectal cancer were evaluated by radiologists at five imaging centers, by two expert reviewers, and by a resident. For each tumor, the minimal distance from the tumor to the circumferential resection margin and the maximum extramural tumor spread were evaluated by the observers. Tumors were classified into early (≤ 5 mm invasion) and advanced (> 5 mm invasion), and margin status was evaluated at the 1- and 5-mm levels.
RESULTS. There was good to very good agreement in classifying tumors as early and advanced (κ = 0.65–0.87), moderate to good agreement concerning circumferential resection margin status at the 1-mm level (k = 0.51–0.76), and fair to good agreement concerning circumferential resection margin status at the 5-mm level (k = 0.37–0.70). It was significantly easier to obtain agreement on the division into early and advanced tumors than on margin status at the 5-mm level for both the hospitals (p = 0.043) and the resident (p = 0.024).
CONCLUSION. Measurements of extramural tumor spread are more reproducible among different observers than are 5-mm distance measurements to the anticipated circumferential resection margin. This factor should be taken into account in the preparation and implementation of guidelines for neoadjuvant therapy for rectal cancer.
Rectal cancer is a common and serious neoplasm. Improvements in both the surgical and the oncologic management of this disease implemented during the last 10–15 years have reduced the incidence of local failure and have improved survival [1–5]. Among prognostic features, circumferential resection margin has emerged as one of the most powerful predictors of outcome. Regardless of local stage of the tumor, the presence of tumor within 1 mm of the surgical circumferential margin is predictive of the development of local recurrence [6–8].
The surgical circumferential resection margin is defined as the surgical cut surface of the connective tissues that encases the rectum. Total mesorectal excision requires precise dissection along the surface of the mesorectal fascia to deliver the rectum encased by its mesorectum containing all the local draining lymph nodes and tumor. MRI has the inherent advantage of consistently depicting the mesorectal fascia and the levator muscles, which form the anticipated surgical circumferential resection margins in total mesenteric excision [9, 10]. Tumor extension to within 1 mm of this fascia, infiltration, and extension beyond this fascia are predictors of subsequent margin involvement.
The clinical guidelines for neoadjuvant therapy vary among countries. According to the Danish national guidelines, among the indications for preoperative therapy, patients with less than 5 mm between the infiltrating tumor border and the mesorectal fascia and levator muscle are offered preoperative long-course chemoirradiation therapy . The recommendations in these guidelines regarding midrectal T3 tumors are based on work by Beets-Tan et al. , who found that a tumor-free margin of at least 1 mm can be predicted with a high degree of certainty when the measured distance on MR images is at least 5 mm, and to some extent on the publications on survival and circumferential resection margin by Wibe et al. .
Preoperative neoadjuvant therapy is more effective and less toxic than postoperative therapy but necessitates accurate preoperative tumor staging for selection of patients who may benefit from neoadjuvant oncologic treatment [13, 14]. The Magnetic Resonance Imaging and Rectal Cancer European Equivalence (MERCURY) study [15, 16] showed that the status of the mesorectal fascia, representing the anticipated circumferential resection margin, and the depth of extramural tumor spread can be predicted with high accuracy with pelvic MRI, pathologic examination being the standard. Hence, a clear margin can be predicted at a 1-mm level with a specificity of 92% (≥ 1 mm to mesorectal fascia indicating clear; < 1 mm, involved or threatened).
Tumors with 5 mm or less of extramural spread regardless of node status have an 85% cancer-specific survival rate compared with poorer-prognosis tumors with more than 5-mm spread, which have only a 54% 5-year cancer-specific survival rate . The MERCURY group also investigated measurements of extramural spread compared with pathologic results and found that the mean difference between measurements was so small that it was considered equivalent. Depth of spread has been validated as an important prognostic indicator [17–20]. The reproducibility of these measurements among radiologists has not been validated.
In this study we aimed to evaluate in a clinical nonexpert setting the reproducibility of two important prognostic staging measurements at pelvic MRI of patients with newly diagnosed rectal cancer, namely, the minimal distance from the invasive tumor to the circumferential resection margin and the predicted depth of extramural tumor spread.
Materials and Methods
This study was approved as a quality assurance project by the local ethics committee. According to Danish law, there was no requirement for informed oral or written consent from the patients. The study was approved by the Danish Data Protection Agency pursuant to the Danish act on storage and processing of personal data. In West Denmark, preoperative pelvic MRI of patients with rectal cancer has been routinely performed since 2001 and, according to the Danish national guidelines, has been mandatory in the workup of rectal cancer since 2002. Two million people reside in West Denmark, and in approximately 360 cases of rectal cancer are diagnosed annually this region .
In 2007, as part of a multidisciplinary development program in West Denmark, a clinical audit of 6 months’ duration was undertaken at the five imaging centers performing rectal MRI in the region. The evaluation included observations of image quality and reporting and interpretation of the tumors in periods 3 months immediately before and 3 months immediately after a multidisciplinary team development course on rectal cancer .
During the course, internationally recognized specialists lectured on topics of multidisciplinary relevance. In total, 31 radiologists attended, and 13 regularly performed pelvic MRI on rectal cancer patients. The radiologists attended an imaging workshop with practical cases and discussed technical performance, sequences, MRI pelvic anatomy, MRI T and N classification, and pitfalls in rectal MRI. At the workshop, minimal requirements in the final MRI report were discussed, and a pro forma template for new diagnoses of rectal cancer was introduced with the recommendation that it be used systematically after the course. A written imaging protocol tailored to the equipment used in the relevant departments was given as handout material  with advice to implement it locally if not already in use. The course was specifically for radiologists and was conducted by one of the authors, who had more than 10 years of experience in pelvic MRI and who had delivered numerous workshops internationally and in the United Kingdom. An overview of the equipment and sequences used in this study appears in Table 1. The examinations were performed with 1.5-T systems (Philips Healthcare, Siemens Healthcare, GE Healthcare) with a phased-array pelvic coil or cardiac coil with the patient in the supine position. The use of antispasmodics was optional.
The team development course was followed by onsite visits to each department by two of the authors. At these onsite visits, the authors participated with the local staff (physicians and radiographers) in imaging of rectal cancer patients to ensure correct implementation of the imaging protocol. Consecutive pelvic MRI examinations of patients with newly diagnosed, biopsy-proved rectal cancer were evaluated by the radiologists at the five centers performing rectal MRI in West Denmark, by two expert reviewers with more than 14 years of practical and scientific experience with pelvic MRI for rectal cancer, and by a second-opinion reviewer, who was a resident with a special interest in rectal MRI.
For each tumor, the minimal distance in millimeters to the mesorectal fascia or levator muscle from the infiltrating border of the main tumor and the maximum extramural spread of tumor outside the lamina muscularis propria in millimeters were evaluated by the observers and reported on specifically developed audit pro forma templates. At the hospitals, the evaluation was performed at PACS workstations (HP Workstation xw6000, Hewlett Packard; Impax software, Agfa Healthcare). The evaluations by the expert reviewers and the second-opinion reviewer were performed with PCs (E-Film software version 3.0, Merge Healthcare; or Univiewer free-ware temporarily available until 2008).
In total, 168 patients with newly diagnosed rectal cancer who had undergone MRI were included in the 6-month period of the audit: 79 patients from 3 months immediately before the multidisciplinary team development course and 89 patients from the 3-month period immediately after the course. All these images were evaluated by the second-opinion reviewer. The 168 examinations were randomly split in two halves, and the two expert reviewers evaluated 84 examinations each. A random sample of 20% of the images (35 patients) was evaluated by both expert reviewers so that information could be acquired on interobserver variation among even very experienced observers. Only measurements from the hospitals after the course (86 patients) were included. The rationale was that the radiologists had learned to assess these tumors and had become familiar with measuring the aforementioned parameters during the course and prospectively reported the parameters as a consequence of the introduction of the audit pro forma template.
Plots and Analysis
Statistical analyses were performed with Stata software (version 11, StataCorp). The minimal distance in millimeters to the mesorectal fascia or levator muscle from the main tumor measured by the radiologists at the five centers was plotted on the x-axis against the corresponding measurements by the expert reviewers on a square root scale on the y-axis. Similar plots were made to compare the second-opinion reviewer with the two expert reviewers, as were plots comparing the two expert reviewers against each other.
Two distances to the mesorectal fascia and levator muscle were evaluated, namely, less than 5 mm to the muscle and less than 1 mm to the fascia. This distance was regarded as the threatened or involved margin. The results were analyzed, and 5 mm or greater and 1 mm or greater were considered to represent clear margins. The 1-mm limit reflected the clinically relevant cutoff point for allocation to neoadjuvant therapy in many countries and has good correlation to margin status at pathologic examination . In Denmark, the 5-mm cutoff point is considered on the basis of work by Beets-Tan et al.  to be the clinically relevant cutoff point for T3 tumors in the midrectum.
On the basis of the measurement of maximum extramural tumor spread in millimeters, the tumors were divided into early (maximum extramural tumor spread, ≤ 5 mm) and advanced (maximum extramural tumor spread, > 5 mm) .
Interobserver agreement on dividing tumors into early and advanced and margin status into involved or threatened or clear was calculated as exact agreement. Kappa statistics were interpreted as follows: < 0.2, poor agreement; 0.21–0.4, fair; 0.41–0.6, moderate; 0.61–0.8, good; 0.81–1.00, very good agreement.
The McNemar test for dependent samples was used to compare agreement on margin status (1- and 5-mm levels) and extramural spread for the observers. Values of p were calculated in the subsamples of patients in whom both relevant measurements of spread and distance to the circumferential resection margin were documented by the observers (e.g., distance to circumferential resection margin was not reported for T2 tumors, tumors in the anal canal, and anterior tumors in the upper rectum). A value of p < 0.05 was regarded as significant .
The cases of 168 patients were evaluated in the 6-month study period. Thirteen consultant radiologists working in five imaging centers were involved in the evaluation of these patients. The centers exhibited variance in caseload (15–53 rectal MRI examinations per center) and number of involved radiologists (one to three radiologists per department) with a mean number of 14 rectal MRI examinations per radiologist evaluated (range, 5–37 examinations) in the 6-month study period. Eighteen percent of the examinations were performed in accordance with the imaging protocol (Table 1) before the course and onsite visits compared with 74% after (p = 0.001) . In total, images from 48% of the examinations were of satisfactory quality. Lack of thin-slice or high-resolution T2-weighted images perpendicular to the tumor was the main deficiency in the unsatisfactory image quality group.
Figure 1 shows plots of the data from the group of radiologists at the hospitals versus expert reviewers, the second-opinion reviewer versus expert reviewers, and first versus second expert reviewer with respect to maximum extramural tumor spread and minimal distance from the infiltrating tumor border to the mesorectal fascia or levator muscle. The plots give an overview of the per-tumor agreement between observers regarding circumferential resection margin and extramural spread. In particular, the plot outlining circumferential resection margin data from the hospitals shows a relatively large number of observations with discordance (gray areas).
Table 2 shows the exact figures with regard to agreement on margin status at the 1- and 5-mm levels for early and advanced tumors and subsamples of tumors in which both extramural spread and distance to the circumferential resection margin were found relevant to report by the involved observers (80 for the radiologists at the hospitals, 32 for the expert reviewers, 162 for the second-opinion reviewer). There was good to very good agreement on the extramural tumor spread criterion (κ = 0.65–0.87), moderate to good agreement on circumferential resection margin status at the 1-mm level (κ = 0.51–0.76), and fair to good agreement on circumferential resection margin status at the 5-mm level (κ = 0.37–0.70).
Table 2 shows that it was significantly easier to obtain agreement on the division into early and advanced tumor than on margin status at the 5-mm level for both the hospitals (p = 0.043) and the second-opinion reviewer (p = 0.024). For the second-opinion reviewer it also was significantly easier to obtain agreement on depth of extramural spread than on margin status at the 1-mm level, but this was not the case for the hospitals. There may be a trend toward better agreement concerning margin status at the 1-mm than the 5-mm level for the hospitals (p = 0.052). No significant differences with regard to these measurements in a subsample of 32 patients were noted for the two expert reviewers.
In Denmark, the Danish Colorectal Cancer Group has agreed on and recommended national clinical guidelines for the allocation of preoperative chemoradiation to rectal cancer patients to ensure that therapy is offered in a standardized manner all over the country. Despite availability of publications from single and multicenter experiences on estimation of extramural tumor extent and proximity of the tumor to the potential circumferential resection margin, little is known about interobserver variability in evaluating these prognostic factors at MRI. Our study showed that measurement of maximum extramural spread is more reproducible between observers with different levels of experience and interests than are measurements of the distance to the anticipated circumferential resection margin represented by the mesorectal fascia or the levator muscle. The study was unique in that we evaluated clinically relevant measurements performed in the field—that is, in a busy hospital practice—and compared the results with measurements performed by highly experienced observers.
Preoperative pelvic MRI of patients with newly diagnosed rectal cancer has been mandatory in Denmark since 2002. The Danish radiologists participating in this study were familiar with the distance measurement to the anticipated circumferential resection margin and were aware of the Danish guidelines concerning allocation to preoperative chemoradiation therapy based on a less than 5-mm distance to the mesorectal fascia or levator muscle. Because of this guideline, observer measurements much larger than 5 mm can be less precise. Therefore, dichotomized interpretation of margin status was used instead of Bland-Altman plots and comparison of correlation coefficients.
The finding of only fair agreement between the expert reviewers and the radiologists at the hospitals concerning margin status at the 5-mm level is of concern in that images in 74% of the examinations of this group of patients were of good technical quality. There seems to be less general agreement about the mesorectal facia, although it is reported to be consistently depicted on MR images, than about the rectal wall and spread into the perirectal fat. The good correlation between the expert reviewers and the second-opinion reviewer shows that it is possible to reproduce margin status at a 5-mm cutoff point even in a test sample with suboptimal image quality in approximately 50% of cases but that doing so may take more expertise and specialization than an average caseload of 14 cases per radiologist per year.
It is an important and potentially serious finding that observers disagree considerably on an issue with possible serious clinical consequences for patients. These results indicate that standardized preoperative treatment allocation is difficult despite guidelines. The data indicate that it may be easier for hospital radiologists to report the 1-mm cutoff level than the 5-mm cutoff level (p = 0.052). This finding seems counterintuitive and warrants explanation. One possible explanation may be that 1 mm may be more reproducible because it almost equals a certain minimal free interface that can be appreciated by the eye and to differentiate this measurement from 0 or 2 mm is more reproducible (involved or not) than to differentiate and measure a distance of, for instance, 4, 5, or 6 mm. Such measurement may even be made differently by the same observer at different times owing to small inaccuracies in the use of the caliper tool and the exact point at which the measurement is made. Another explanation could be that the mesorectum is often asymmetric, usually with a shorter anteroposterior diameter than left-right diameter, and the rectum may lie asymmetrically in this oval cylinder of perirectal fat. It is therefore of great importance that the radiologist judge the morphologic features of the tumor and the location of the infiltrating border correctly to choose the correct site for measurement of the shortest distance from tumor infiltration to fascia (Figs. 2A, 2B, 3A, 3B, 4A, and 4B). This detailed interpretation of tumor morphology may not be that critical if the distance limit is less than 1 mm, as was the case in the MERCURY study, but it becomes of great importance for evaluation at the 5-mm level, as in the Danish national guidelines.
Circumferential resection margin status is a major predictor of outcome from rectal cancer. It has been suggested that circumferential resection margin status is even more informative in treatment planning than is T category . A clear margin reduces the risk of local recurrence and increases the chance of survival [6–8, 26]. However, with a local failure rate of 5–10% and a 5-year survival rate of 50–60%, it is evident that distant failure has become the major cause of mortality among these patients . T3 tumors with extramural tumor spread exceeding 5 mm are more often associated with lymph node involvement than T3 tumors with extramural tumor spread of 5 mm or less. The same applies to vascular spread and hence the risk of distant metastasis.
Measurement of extramural tumor spread is not a part of the Danish guidelines and has therefore not been measured routinely before but was introduced as a part of this audit. Despite the fact that the participating radiologists were unfamiliar with this measurement before the development course, we found that this quantitative measurement was well reproduced among radiologists. This measurement is predictive of the presence of other risk factors, such as lymph node involvement and vascular invasion, which at rectal MRI require qualitative measurements that are difficult to predict accurately and to reproduce among observers [22, 28–31].
In neoadjuvant chemoradiation therapy, radiation is directed against the local disease, and the chemotherapy is generally a radiation sensitizer but also is directed against metastatic spread. Different clinical guidelines and discrepancies between countries with regard to recommendations may to some extent be attributable to the different weighting of the risks of local and distant failure [32, 33]. In this context it is important to consider that there may be a discrepancy between what would be desirable prediction accuracy on MR images and what can actually be predicted with high reproducibility. Guidelines for allocation to chemoradiotherapy ought to be developed in accordance with this reality.
The MERCURY study  showed that tumor spread can be estimated accurately with MRI in comparison with pathologic results. Our results show that this prediction can be reliably reproduced among radiologists. The Danish guidelines on allocation of midrectal T3 tumors to chemoradiotherapy are mainly based on results of a retrospective comparative study by Beets-Tan et al. . Our data on the reproducibility of distance measurements and data from the MERCURY study, in which pathologic examination was the standard, indicate that these guidelines should be altered to facilitate standardization in treatment allocation and to reduce overtreatment with preoperative chemoradiotherapy because the 5-mm distance may be difficult to interpret and reproduce among observers.
Measurement of extramural tumor spread is more reproducible among radiologists than is measurement of a 5-mm distance to the anticipated circumferential resection margin. This factor should be taken into account in the preparation and implementation of guidelines for neoadjuvant therapy for rectal cancer.
We thank the radiologists and radiographers at the participating hospitals in the county of West Denmark for their cooperation in this study.
Financially supported by the County of West Denmark.
Heald RJ, Moran BJ, Ryall RD, Sexton R, MacFarlane JK. Rectal cancer: the Basingstoke experience of total mesorectal excision, 1978-1997. Arch Surg 1998; 133:894–899
Martling AL, Holm T, Rutqvist LE, Moran BJ, Heald RJ, Cedemark B. Effect of a surgical training programme on outcome of rectal cancer in the County of Stockholm. Stockholm Colorectal Cancer Study Group, Basingstoke Bowel Cancer Research Project. Lancet 2000; 356:93–96
Kapiteijn E, Putter H, van de Velde CJ. Impact of the introduction and training of total mesorectal excision on recurrence and survival in rectal cancer in The Netherlands. Br J Surg 2002; 89:1142–1149
Peeters KC, Marijnen CA, Nagtegaal ID, et al. The TME trial after a median follow-up of 6 years: increased local control but no survival benefit in irradiated patients with resectable rectal carcinoma. Ann Surg 2007; 246:693–701
Quirke P, Durdey P, Dixon MF, Williams NS. Local recurrence of rectal adenocarcinoma due to inadequate surgical resection: histopathological study of lateral tumour spread and surgical excision. Lancet 1986; 2:996–999
Brown G, Radcliffe AG, Newcombe RG, Dallimore NS, Bourne MW, Williams GT. Preoperative assessment of prognostic factors in rectal cancer using high-resolution magnetic resonance imaging. Br J Surg 2003; 90:355–364
Pedersen BG, Blomqvist L, Brown G, Fenger-Gron M, Moran B, Laurberg S. Postgraduate multidisciplinary development program: impact on the interpretation of pelvic MRI in patients with rectal cancer: a clinical audit in West Denmark. Dis Colon Rectum 2011; 54:328–334
Wolberink SV, Beets-Tan RG, Nagtegaal ID, Wiggers T. Preoperative assessment of the circumferential margin in rectal cancer is more informative in treatment planning than the T stage. Tech Coloproctol 2006; 10:171–176
Nagtegaal ID, Marijnen CA, Kranenbarg EK, van de Velde CJ, van Krieken JH. Circumferential margin involvement is still an important predictor of local recurrence in rectal carcinoma: not one millimeter but two millimeters is the limit. Am J Surg Pathol 2002; 26:350–357
Koh DM, George C, Temple L, et al. Diagnostic accuracy of nodal enhancement pattern of rectal cancer at MRI enhanced with ultrasmall super-paramagnetic iron oxide: findings in pathologically matched mesorectal lymph nodes. AJR 2010; 194:1530; W505–W513
Matsuoka H, Nakamura A, Sugiyama M, Hachiya J, Atomi Y, Masaki T. MRI diagnosis of mesorectal lymph node metastasis in patients with rectal carcinoma. what is the optimal criterion? Anticancer Res 2004; 24:4097–4101