| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Original Research |
1 Department of Radiological Sciences, Division of Diagnostic Imaging, St. Jude
Children's Research Hospital, 332 N Lauderdale St., Memphis, TN 38105.
2 Department of Medicine, University of Tennessee College of Medicine, Memphis,
TN.
Received October 12, 2004;
accepted after revision January 31, 2005.
Address correspondence to S. C. Kaste.
Abstract
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
MATERIALS AND METHODS. We conducted a retrospective analysis of MRI studies of the knee performed in a single institution between April 1994 and July 2003. Knee osteonecrosis was identified in 168 children with a primary diagnosis of hematologic malignancy. This substantial number prompted us to design a staging system for use with pediatric patients. To assess interobserver reliability of two primary observers in using the system, they reviewed and interpreted the same 36 imaging studies of randomly chosen patients. For the assessment of intraobserver reproducibility, each observer rereviewed 16 studies. A senior observer coded potential causes of disagreement between the primary observers.
RESULTS. Interobserver agreement was substantial: the kappa value was 0.66 (95% confidence interval [CI], 0.58-0.75) in locations where the observers had to record only the presence or absence of a lesion, and the weighted kappa value was 0.65 (95% CI, 0.59-0.72) in locations where they had to classify the extent of involvement. The presence of marrow edema, punctate foci of altered signal, and mottled marrow changes was associated with a higher level of disagreement between the primary observers.
CONCLUSION. Our proposed classification system, developed specifically for use with MRI, was used with substantial intra- and interobserver agreement. We think its use can contribute to a standardized approach to the interpretation of MRI findings in pediatric osteonecrosis of the knee.
Keywords: knee • leukemia • lymphoma • MRI • musculoskeletal imaging • oncology • osteonecrosis • pediatric imaging
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
The substantial number of patients found to have osteonecrosis of the knee prompted us to study its development and to design an osteonecrosis staging system for use in pediatric patients. We developed an MRI-based system because this imaging technique can detect early osteonecrotic changes well before they would become visible on radiography [15-18]. We eventually plan to develop an algorithm for predicting the risk of functional deterioration of the knee in these patients [19].
In addition to the evidence of leukemia itself, bone marrow abnormalities resulting from disease or therapy are evident on the MR images of children treated for leukemia [20-22]. Of these abnormalities, osteonecrosis is probably associated with the most serious risk of morbidity, so it is important to distinguish osteonecrosis from other pathologic findings.
In this article, we describe a new MRI-based system for classifying osteonecrotic damage of the knee in children treated for hematologic malignancies. By testing the two primary observers' evaluations of randomly selected imaging studies, we determined whether this system can be used with an acceptable level of intra- and interobserver agreement. Using additional information provided by a senior observer, we also estimated the influence of factors that may have led to interobserver variability in classifying the extent of disease.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Evidence of knee osteonecrosis was found in 168 patients. For the measurement of interobserver reliability, 36 MRI studies belonging to individual patients were randomly selected from those of 168 children and adolescents with hematologic malignancies and evidence of knee osteonecrosis. The two primary observers' evaluations were compared to measure their level of agreement in determining the presence and extent of osteonecrosis. Thirty-two of these 36 imaging studies (16 for each observer) were used for intraobserver reliability assessment. Intraobserver reliability was measured by comparing each observer's interpretations of second reviews of 16 imaging studies with their interpretations of the first reviews of the studies. To minimize recall bias, we asked the observers to reinterpret small batches of images over a period of a year, starting at least 1 month after completion of the first reviews.
To determine factors that may have led to interobserver disagreement in identifying and classifying osteonecrotic lesions, the senior observer reviewed and interpreted the same 36 MRI studies reviewed by the primary observers. The senior observer also recorded evidence of nonosteonecrotic abnormalities of the knee. The purpose of the senior observer's additional review was to identify pathologic findings that may complicate the diagnosis of osteonecrotic lesions in the patients.
MRI Evaluation
MRI evaluation of the knee consisted of coronal unenhanced T1-weighted
imaging (TR/TE, 400/14), coronal STIR imaging (3,550/28), and sagittal fast
low-angle shot (FLASH) 2D imaging (588/10.5). All MRI examinations were
performed on one of the following 1.5-T scanners: Helicon, Vision, or Symphony
(Siemens Medical Solutions) using a torso phased-array coil. The scanning
parameters included a slice thickness of 5 mm and interslice gap of 1 mm.
Image Interpretation
The primary observers completed a data collection form for each patient;
they recorded their observations for the right and left knees separately. The
status of the physes (open or closed) was recorded for the distal femur and
proximal tibia. Eight locations per knee were evaluated for the presence of an
osteonecrotic lesion: distal femoral diaphysis, distal femoral metaphysis,
medial and lateral distal femoral epiphyses, medial and lateral proximal
tibial epiphyses, proximal tibial metaphysis, and proximal tibial diaphysis,
producing 2,688 (8 x 2 x 168) areas of observation. The three
observers used the same criteria to define a lesion—that is, a
geographic area of decreased signal on T1-weighted images and increased signal
on STIR images, as shown in Figures
1A and
1B of part 2
[19] of this series, which
appears after this article.
|
|
The extent of necrosis and the involvement of the articular surface have been linked to a progressive course of osteonecrosis in several studies [8, 23, 24]. We developed the classification method for use in this study to gain insight into the association between clinical manifestations of osteonecrosis and MRI findings of osteonecrosis [19] and to facilitate longitudinal observations of the development of osteonecrosis.
Statistical Analysis
Thirty-six patients were randomly selected for estimation of interobserver
reliability. The number 36 was chosen to provide an adequate sample size and
thus adequate power for statistical tests to detect departures from the null
hypothesis of very weak association within the population under study. At an
alpha level of 0.05, the sample provides 80% power to discriminate between a
weak association of 0.40 and a strong association of 0.70. Additional details
about the statistical methods are given below.
Observer reliability—Kappa and weighted kappa statistics
[25,
26] with linear sets of
weights were calculated to measure interobserver and intraobserver
reliability. The simple Cohen kappa was used to analyze agreement of findings
related to diaphyses and metaphyses. Weighted kappa
[26] was used for the findings
related to epiphyses, which required choosing from multiple categories.
Guidelines suggested by Landis and Koch
[25] were used to describe the
strength of agreement based on the kappa value. To make the disagreement
between nonadjacent categories, such as lesions occupying less than 25% of
articular surface and lesions occupying more than 50% of articular surface,
more apparent, we used a linear weighting scheme:
 |
Causes of discrepancy—Using additional information provided by the senior observer, we estimated the influence of nonosteonecrotic pathologic findings on the lack of agreement between the primary observers in identifying the presence of an osteonecrotic lesion and in determining osteonecrosis extent. Kappa statistics and Fisher's exact test were used.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Interobserver Agreement
Interobserver agreement was substantial (overall proportion of agreement
[Po] = 83%, 
= 0.66, 95% confidence interval [CI] =
0.58-0.75) when the assessment included only the indication of the presence or
absence of an osteonecrotic lesion in the metaphyses and diaphyses
(Table 1). Assessment of the
epiphyses required choosing one of five possible categories of involvement
(Table 2). The results were as
follows: Po = 66% and = 0.48. After linear weighting, the
results were a weighted kappa value (w) of 0.65 and a 95% CI
of 0.59-0.72, which signify a substantial level of agreement. Observers showed
excellent agreement in their classification of the status of the physes (open
or closed): Po = 100% and = 1.0.
|
|
Intraobserver Agreement
The primary observers showed substantial intraobserver agreement in their
assessment of metaphyses and diaphyses: observer A, Po = 90%,
= 0.78, 95% CI = 0.67-0.89; and observer B, Po = 83%,
= 0.66, 95% CI = 0.54-0.79 (Table
3). Intraobserver agreement was also substantial in the assessment
of epiphyses: observer A, Po = 70%, = 0.52,
w = 0.65, 95% CI of w = 0.55-0.75; and
observer B, Po = 78%, = 0.67, w = 0.80,
95% CI of w = 0.74-0.86
(Table 4). Intraobserver
agreement was perfect in the classification of the openness of physes:
observers A and B, Po = 100% and = 1.0.
|
|
Causes of Discrepancy
The most common nonosteonecrotic pathologic findings recorded by the senior
observer were bone marrow edema (Figs.
1A and
1B), which was found in 168 of
568 evaluated locations; punctate lesions of uncertain cause (Figs.
2A and
2B), which were found in 105 of
568 evaluated locations; and mottled marrow changes (Figs.
3A and
3B), which were found in only
the metaphyses and diaphyses (54 of 280 locations). Edema was found more often
with osteonecrotic lesions than without them (48% [119/246] vs 15% [49/322],
respectively; p 
0.001). Punctate foci of altered signal were
found slightly more frequently with osteonecrotic lesions than without them
(21% [52/246] vs 16% [53/322], respectively; p = 0.24). Mottled
marrow changes, which were found in only metaphyseal and diaphyseal regions,
were associated with the absence of osteonecrotic lesions (27% [49/179] vs 5%
[5/101], respectively; p 0.001).
|
|
|
|
In locations where at least one nonosteonecrotic pathologic finding was
present, interobserver agreement declined substantially
(Table 5). In metaphyses and
diaphyses, the observed Po declined from 90% for locations without
such findings to 83% for locations with edema, 60% for locations with punctate
foci of altered signal, and 63% for locations with mottled marrow changes
(Table 5). Changes in the index
of agreement corrected by chance () were even more apparent: Agreement
declined from substantial ( = 0.78) for locations where no edema,
punctate foci, or mottled marrow changes were present to moderate ( =
0.51) for locations where marrow edema was found and to fair ( = 0.26)
for locations where punctate foci of altered signal were present. It was also
fair ( = 0.23) for locations where mottled marrow changes were present
(Table 5). Similar changes in
the level of agreement were noticed in epiphyses: The proportion of agreement
and w declined from 80% and 0.81 (substantial agreement) for
locations where no edema or punctate foci of altered signal were present to
55% and 0.53 (moderate agreement), respectively, for locations where edema was
present, and to 50% and 0.24 (fair agreement), respectively, for locations
where punctate foci of altered signal were present.
|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
We chose a method for evaluation of lesion size for a combination of practical and clinical reasons. Other studies of osteonecrosis have used a more precise method for the measurement of osteonecrotic lesion size—most notably, that of Mont et al. [24], who used methods developed by Lotke and Ecker [28] and Kerboul et al. [29] for calculation of the lesion size. Although accurate, these methods are more suitable for studying single lesions in selected patients. These methods are complicated to use in a practical setting, especially in a study that requires routine monitoring of patients being treated for leukemia. On the other hand, we wanted to emphasize the clinical importance of articular surface involvement, which has been linked to a worse outcome in at least one study of knee osteonecrosis in pediatric leukemia patients [23]. When involvement of the articular surface occurs, it puts the knee at risk for further progression of osteonecrosis, development of arthritis, and functional deterioration. The relative size of the area of the articular surface involved in osteonecrosis may be important in influencing clinical symptoms and tracking disease progression. As a result, we designed a staging system that is descriptive and easy to use. The observed interobserver agreement and intraobserver agreement in using the present MRI-based staging system for osteonecrotic lesions on the knee were substantial and within a previously reported range for imaging discrepancy [30-32]. Therefore, we think that use of the present system can contribute to a standardized approach to the interpretation of MRI findings of osteonecrosis of the knee in pediatric patients.
In the current study, a lack of interobserver agreement was most often due to differences in defining an osteonecrotic lesion. Various osteonecrosis-related abnormalities may be present in the same locations where marrow changes related to leukemia and chemotherapy are found. As a result, obtaining agreement about the presence of an osteonecrotic lesion was sometimes difficult. The prevalence and locations of the nonosteonecrotic abnormalities we found warrant special attention in future studies of osteonecrosis. Edema was found primarily in locations where osteonecrotic lesions were also found. Punctate foci of altered signal were found with almost equal frequency in locations with and without lesions. Mottled marrow changes were found only in the metaphyses and diaphyses, predominantly in locations without osteonecrotic lesions. Although analyzing the origin of these abnormalities was beyond the scope of our current study, investigating this issue in future studies may provide insight into the development of osteonecrotic lesions.
The limitations of this study include the modest number of observers available for thorough validation of the proposed classification system. The observers' familiarity with osteonecrosis of the knee may have contributed to the substantial level of agreement in identifying and describing osteonecrotic lesions. In actual practice, radiologists may have more trouble correctly identifying osteonecrotic lesions. On the other hand, the importance of confounding factors that make the interpretation more difficult (the presence of edema, punctate foci of altered signal, and mottled marrow changes) was not fully understood until the conclusion of this study. Awareness of these factors and specific training before the start of similar investigations should further improve interobserver reliability and intraobserver reproducibility. However, at present we cannot make a quantitative prediction of such improvement. In this study, the imaging was not standardized according to the phase of treatment; however, we address this issue in an ongoing prospective study of osteonecrosis in leukemia patients being conducted at our institution.
In conclusion, the system here proposed for the classification of osteonecrosis was used with a substantial level of observer agreement and will be used at our institution to prospectively study the development of osteonecrosis in survivors of childhood leukemia. The diagnosis of osteonecrosis of the knee in these and other children with a hematologic malignancy may be complicated when marrow edema, punctate lesions of uncertain cause, or changes of marrow recovery are present. These abnormalities were found in nearly half of the anatomic locations examined in our study. Differential diagnosis of osteonecrosis with these pathologic findings should be emphasized in radiologists' training.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
This article has been cited by other articles:
|
|
 |
|
  E. J. Karimova, S. N. Rai, D. Ingle, A. C. Ralph, X. Deng, M. D. Neel, S. C. Howard, C.-H. Pui, and S. C. Kaste MRI of Knee Osteonecrosis in Children with Leukemia and Lymphoma: Part 2, Clinical and Imaging Patterns Am. J. Roentgenol., February 1, 2006; 186(2): 477 - 482. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
