DOI:10.2214/AJR.07.3558
AJR 2008; 191:190-197
© American Roentgen Ray Society
Radiologists' Agreement When Using a 10-Point Scale to Report Abdominal Radiographic Findings of Necrotizing Enterocolitis in Neonates and Infants
Courtney A. Coursey1,
Caroline L. Hollingsworth1,
Ana M. Gaca1,
Charles Maxfield1,
David DeLong2 and
George Bisset, III1
1 Department of Radiology, Duke University Medical Center, Erwin Rd., Box 3808,
Durham, NC 27710.
2 Department of Biostatistics and Bioinformatics, Duke University Medical
Center, Durham, NC.
Received December 17, 2007;
accepted after revision January 14, 2008.
Address correspondence to C. L. Hollingsworth
(holli016{at}mc.duke.edu).
CME
This article is available for CME credit. See
www.arrs.org
for more information.
Abstract
OBJECTIVE. The purpose of this study was to evaluate radiologists'
agreement when using a 10-point scale of abnormal findings designed to
standardize reporting of abdominal radiographs in neonates or infants with
suspected necrotizing enterocolitis.
MATERIALS AND METHODS. A 10-point scale of radiographic findings was
devised at our institution and was in use for approximately 18 months before
the initiation of this study. After institutional review board approval, 88
abdominal radiographs (anteroposterior and cross-table lateral) were randomly
selected for review, allowing for an equal distribution of examinations
throughout the scale according to the original examination report. The mean
age of the patients in the total study population was 24.9 days (range,
0–56 days); 61 patients (47.3%) were girls and 68 (52.7%) were boys.
Four pediatric radiologists having 20, 13, 7, and 5 years of experience scored
images twice at least 4 weeks apart according to the scale, which was designed
to characterize certainty and severity of disease in neonates and infants with
possible necrotizing enterocolitis. Interobserver and intraobserver agreement
was assessed by applying weighted kappa statistics. Operative and pathology
reports were reviewed.
RESULTS. The average intraobserver weighted kappa value was 0.792
(SD, 0.025; range, 0.635–0.946). The average interobserver weighted
kappa value was 0.665 (SD, 0.035, range, 0.574–0.898).
CONCLUSION. Substantial intraobserver and interobserver agreement
was found when radiologists used a 10-point scale to report abnormal findings
on abdominal radiographs in neonates or infants with suspected necrotizing
enterocolitis. This scale warrants further evaluation as a potentially useful
clinical tool.
Keywords: enterocolitis • necrotizing enterocolitis • observer agreement • pediatric imaging • radiography • radiologists • radiology reporting systems • scale of abnormal radiographic findings of necrotizing
enterocolitis
Introduction
Necrotizing enterocolitis is an acquired inflammatory disease of the
gastrointestinal tract and one of the most common abdominal emergencies in the
premature neonate [1,
2]. Abdominal radiographs play
an important role in the evaluation of patients with suspected necrotizing
enterocolitis. These patients are often placed on "necrotizing
enterocolitis watch," which includes bowel rest, antibiotics, and
abdominal radiography every 6–8 hours. The findings in these abdominal
radiographs can alter patient management and even indicate the need for
surgery. Therefore, it is important that these findings be communicated to the
referring neonatologist in a clear and consistent manner.
The language used in radiology reports varies from radiologist to
radiologist [3]. In addition,
terms used to indicate level of diagnostic certainty such as "suggestive
of" and "suspicious for" have different meanings for both
radiologists and nonradiologists
[4]. The phrase
"nonspecific abdominal gas pattern" may indicate a normal
condition to some radiologists and referring physicians and an abnormality to
others [5]. Variability in
reporting abdominal radiographic findings in neonates and infants with
suspected necrotizing enterocolitis can make it challenging for the referring
neonatologist to understand the diagnostic certainty of the radiologist and to
modify the treatment plan accordingly.
Standardized lexicons such as the American College of Radiology (ACR)
Breast Imaging Reporting and Data System (BI-RADS)
[6] have been devised to
decrease confusion in terminology and disposition in mammography reporting.
Training in BI-RADS has been shown to improve observer agreement with the
consensus opinion of experienced breast imagers
[7].
With the goal of standardizing reporting of abnormal radiographic findings
in neonates and infants with suspected necrotizing enterocolitis at our
institution, a 10-point scale of abnormal radiographic findings was devised
(Table 1). Increasing numbers
on the scale are meant to reflect increasing certainty that a patient has
necrotizing enterocolitis and increasing concern regarding the severity of the
patient's disease. It was not the purpose of our study to validate this
correlation.
View this table:
[in this window]
[in a new window]
|
TABLE 1: Duke Abdominal Assessment Scale (DAAS) of Abnormal Radiographic Findings
in Neonates and Infants with Clinically Suspected Necrotizing
Enterocolitis
|
|
Several prior investigations have evaluated the role of abdominal
radiographs in the diagnosis and management of neonates and infants with
necrotizing enterocolitis
[8–10].
Those studies, which did not use a standardization tool such as a scale of
abnormal findings, found poor interobserver and intraobserver agreement in
film interpretation
[8–10].
The purpose of this study was to evaluate radiologists' intraobserver and
interobserver agreement when using this 10-point scale designed to standardize
reporting of abdominal radiographs in neonates and infants with suspected
necrotizing enterocolitis.
Materials and Methods
Institutional review board approval was obtained for this HIPAA-compliant
study, and a waiver of informed consent was granted. All two-view
(anteroposterior and cross-table lateral) abdominal radiographs obtained in
the neonatal intensive care unit between July 1, 2005, and June 30, 2006, were
reviewed for study indication. Abdominal radio graphic series satisfying the
following criteria were selected for inclusion: patient 2 months old or
younger at time of study, frontal and cross-table lateral views obtained,
indications for study were "evaluate for necrotizing
enterocolitis," "evaluate pneumatosis," "disten
tion," "abnormal loops," "bloody stools," or
"evaluate for perforation."
Of 2,666 two-view abdominal examinations obtained from 297 patients in the
neonatal intensive care unit during the study period, 479 abdominal
radiographic series satisfied these criteria. The Duke Abdominal Assessment
Scale (DAAS) (Table 1) score
reported in the radiology report of the original examination was recorded.
From these 479 cases, nine were randomly selected for review in each category
determined by the original report of the DAAS. Cases selected as sample cases
came from a randomized list of all cases in the study period that satisfied
inclusion criteria, sorted by the original score. For example, all
examinations given an original score of 0 were grouped together but listed in
random order. This was also done for all cases with an original score of 1, 2,
3,... 10. Then the first nine randomly ordered cases from each category (score
1, score 2,...) were reviewed, and a sample case was selected from each
category. The 11 sample paired (anteroposterior and cross-table lateral)
abdominal radiographs served as the basis for score assignment of 0 (normal
examination) and 1 (mild distention) through 10 (pneumoperitoneum) (Fig.
1A,
1B,
1C,
1D,
1E,
1F,
1G,
1H,
1I,
1J,
1K,
1L,
1M,
1N,
1O,
1P,
1Q,
1R,
1S,
1T,
1U,
1V). The remaining eight cases
in each category were included in the study, resulting in a total of 88 cases
from 49 patients included in the study for review. This method was chosen to
ensure an adequate sample size for each DAAS score. Randomness was achieved by
using the Excel (Microsoft) random number generator function.
View larger version (137K):
[in this window]
[in a new window]
[as a PowerPoint slide]
|
Fig. 1O —Example radiographs reviewed by each observer.
Anteroposterior (O) and cross-table lateral (P) examples of
score 7, fixed or persistent dilatation of bowel loops, on radiographs
obtained approximately 24 hours apart.
|
|
View larger version (137K):
[in this window]
[in a new window]
[as a PowerPoint slide]
|
Fig. 1P —Example radiographs reviewed by each observer.
Anteroposterior (O) and cross-table lateral (P) examples of
score 7, fixed or persistent dilatation of bowel loops, on radiographs
obtained approximately 24 hours apart.
|
|
Clinical data, including patient sex, age at time of study, study
indication provided, and any abdominal surgical procedures, were recorded but
were not available to study participants during review of cases. Four
pediatric radiologists with subspecialty training in pediatric radiology and
20, 13, 7, and 5 years of experience were recruited as study participants. The
study participants were blinded to the distribution of DAAS scores among the
88 cases submitted for their review.
The sample abdominal radiographs and the accompanying scores were reviewed
by each of the study participants before interpretation of the study
examinations. The 88 pairs of abdominal radio graphs included in the study
were then interpreted twice by all four participants at least 4 weeks apart.
Radio graphs were interpreted using a PACS (Centricity, GE Healthcare)
currently in use for clinical interpretation at our institution under standard
reading room ambient lighting. All patient identifiers were concealed, and a
case number was randomly assigned to each examination. Study participants were
blinded to the distribution of cases across all categories (0–10) during
the review process. Each study participant reviewed and interpreted case
examinations in the same sequence (cases 1–88). Study participants were
blinded to the original report, scores of other participants, and their own
previously recorded scores during the study. To simulate interpretation of a
clinical examination, the most recent prior abdominal radiographic series
(anteroposterior and cross-table lateral) was also provided for review during
interpretation of the study cases. Each study participant recorded a single
score using the DAAS for each case. For example, a score of 1 indicates mild
diffuse distention of bowel loops, a score of 3 indicates focal distention of
bowel loops, and a score of 7 indicates fixed (unchanging) loops of
bowelcompared with the prior examination.
Data analysis was performed by a statistician using version 9.1 of the SAS
software system (SAS), which calculated weighted kappa statis tics. Kappa
values of reader agreement with the initial score, intraobserver agreement,
and interobserver agreement were computed. Stand ard errors and related
statistical tests were computed by means of the statistical jackknife.
Intraobserver agree ment was determined for the scoring system by comparing
data from the same observer at two reading sessions. Interobserver agreement
was determined by comparing data between readers on either occasion. Thus,
four intraobserver and 24 interobserver measures of agreement could be
derived. Kappa coefficients were calculated as indicators of intra- and inter
observer agreement. The level of agreement between readers was characterized
by weighted kappa values (
w), which provide a measure of
inter- and intra observer agreement adjusted for chance of agree ment.
Standard 95% CIs were used for testing
[11].
Results
The 479 abdominal radiographs during the study period that satisfied
inclusion criteria were obtained from 129 neonates and infants. The mean age
of the patients in the total study population was 24.9 days (range, 0–56
days); 61 patients (47.3%) were girls and 68 (52.7%) were boys. An average of
3.4 abdominal radiographic series that satisfied our criteria were obtained
per patient (range, 1–23 series).
The four weighted kappa values for intraobserver agreement ranged from
0.635 to 0.946 (Table 2). The
24 weighted kappa values for interobserver agreement ranged from 0.574 to
0.898, and those for agreement with the score in the original radiology report
ranged from 0.562 to 0.757 (Table
2). According to Landis and Koch
[12], a kappa statistic of
0.80–1.00 reflects almost perfect agreement, 0.61–0.80 reflects
substantial agreement, and 0.41–0.60 reflects moderate agreement. Our
results indicate substantial intraobserver agree ment, substantial
interobserver agreement, and substantial agreement with the score in the
original radiology report.
All four reviewers agreed on 24 of 88 cases in the first interpretation
session. Three of four reviewers agreed on an additional 23 cases. As
illustrated in Table 3, reader
agreement was greatest for DAAS 9 (portal venous gas) and DAAS 10
(pneumoperitoneum). Reader agreement was poorest for DAAS 4 (separation or
focal thickening of bowel loops) and DAAS 6 (possible pneumatosis with other
abnormal findings).
Discussion
Abdominal radiographs are a standard part of the evaluation of patients
with suspected necrotizing enterocolitis. The findings on these abdominal
radiographs may guide treatment and can indicate the need for surgery.
Communicating these findings to the referring neonatologist in a clear and
consistent manner is an integral part of providing prompt, efficient patient
care. In the absence of a standardized lexicon, the language of radiology
reports may vary substantially from radiologist to radiologist.
In an analysis of chest radiography reports obtained from 822 patients,
Sobel et al. [3] found 23
synonyms for reporting the presence of an abnormality and 30 synonyms for
reporting that an abnormality may or may not be present. Khorasani et al.
[4] found the perceived
diagnostic certainty of phrases such as "consistent with" and
"suspicious for" to be quite variable among radiologists and
nonradiologists. Khorasani et al. asked 45 staff radiologists and 158
referring physicians to rank 15 commonly used phrases meant to convey
diagnostic certainty from 1 (most certain) to 15 (least certain)
[4]. Agreement among both
radiologists and nonradiologists was excellent for the term "diagnostic
of" but was poor for the remaining 14 terms
[4]. For example, even among
radiologists, the term "unlikely" was ranked from 2 to 15 and the
term "compatible with" was ranked from 3 to 15
[4].
Lack of clarity and consistency in radiology reporting makes it more
challenging for the referring clinician to incorporate radiographic findings
into his or her treatment algorithm. We undertook this project as the initial
step in validating a standardized lexicon for the reporting of abnormal
radiographic findings in neonates and infants with suspected necrotizing
enterocolitis.
Because no single physical, laboratory, or radiographic finding is highly
sensitive and specific for the diagnosis of necrotizing enterocolitis, the
diagnosis tends to be based on a composite of clinical and radiographic
findings. Various imaging techniques, including sonography
[13,
14], MRI
[14], contrast enema
[15], and CT
[16], have been postulated to
add diagnostic value, but none has proven consistently reliable. The use of
sonography is increasing, and some studies have shown the merit of this
technique both in the evaluation of bowel necrosis
[17] and in the detection of
portal venous gas [18].
However, abdominal radiography remains the most widely used diagnostic imaging
technique in the evaluation of neonates and infants with suspected necrotizing
enterocolitis. Although the presence of pneumatosis intestinalis on abdominal
radiography in the neonatal intensive care unit is almost always indicative of
necrotizing enterocolitis, other early and late radiographic findings such as
bowel dilation, regional paucity of bowel gas, and separation of bowel loops
are suggestive but nonspecific
[19]. Pneumoperitoneum is the
only universally agreed on radiographic sign that mandates surgical
intervention
[19–21].
However, even this sign is present in only 50–75% of all neonates and
infants with bowel perforation due to necrotizing enterocolitis
[22–24].
The DAAS was formulated to reflect progressive disease and increased
certainty of the diagnosis of necrotizing enterocolitis with increasing
numeric score. Generalized bowel dilatation, both mild and moderate, is
thought to be one of the earliest radiographic signs of necrotizing
enterocolitis (scores 1 and 2). Prior investigations have shown radiographic
evidence of bowel dilatation in 75%
[21] to more than 90% of cases
of necrotizing enterocolitis
[19,
23]. Focal dilation is thought
to reflect more advanced disease (score 3)
[23]. Focal, separated, or
featureless loops of bowel (scores 4 and 5) often reflect even more advanced
necrotizing enterocolitis
[19]. Because distinguishing
pneumatosis from fecal matter mixed with air is frequently a challenging task,
we included categories reflecting varying degrees of certainty regarding the
presence of pneumatosis (scores 2, 6, and 8). The persistent "loop
sign" (score 7), first described in 1978 by Wexler
[25], refers to a loop of
bowel that is relatively unchanged over a 24- to 36-hour period. Fixed loops
of bowel reflect nonperistaltic bowel secondary to necrosis of mucosa,
submucosa, and serosa [25].
This finding is thought to be a sign of more advanced necrotizing
enterocolitis and should raise the suspicion of impending perforation (score
7). In neonates and infants with pneumatosis intestinalis, air can dissect
into the portal veins, resulting in portal venous gas
[26] (score 9).
Pneumoperitoneum, reflecting bowel perforation, is an indication for surgical
intervention
[20–22]
(score 10).
Not all cases of necrotizing enterocolitis progress in such a fashion. In
fact, seven of the 12 patients who developed pneumoperitoneum in our study had
no evidence of pneumatosis immediately before developing pneumoperitoneum.
However, the use of a standardized lexicon such as the DAAS allows the
communication of abnormal and potentially worrisome findings to the clinical
service caring for these infants in a clear, precise manner. The aim of this
study was to validate a radiographic scoring system for the interpretation of
abdominal radiographs in neonates and infants with clinically suspected
necrotizing enterocolitis.
Problems in designing scoring systems include difficulty in building a
reproducible classification that can describe findings in a progressive and
additive way. Pathologic changes in necrotizing enterocolitis do not
necessarily follow a standardized pattern; thus, patients may present with
some abnormalities from a higher score in the classification system without
orderly progression through the lower scores. A key objective in the
validation process is reproducibility and consistency because these attributes
are an integral part of the usefulness of any scoring system. This
investigation served to initiate validation of the DAAS by evaluation of
intraobserver and interobserver agreement when using the DAAS 10-point scale.
This preliminary investigation supports that the DAAS provides consistency in
the reporting of abnormal radiographic findings in neonates and infants with
clinically suspected necrotizing enterocolitis.
Table 3 depicts the median
DAAS score for each study case relative to the individual score assigned by
each reader. DAAS scores of 0 and 1 (normal gas pattern vs mild distention)
show overlap with each other, indicating a greater divergence of the median
score. DAAS scores of 3 and 4 (focal moderate distention vs separation or
focal thickening of loops of bowel) also have relatively increased spread of
the median score. In addition, some overlap in cases given a DAAS score of 6
and 8 (possible pneumatosis vs highly probable or definite pneumatosis)
occurred because of different degrees of certainty observers had regarding the
presence of pneumatosis. Relatively less variability was seen in the median
scores of 7, 9, and 10.
A prior investigation by Mata and Rosengart
[9] showed poor interobserver
agreement when 10 observers (six pediatric radiologists, three staff
neonatologists, and one senior neonatology fellow) were asked to evaluate 17
abdominal radiographic series and decide whether the radiograph
(anteroposterior film) or set of films (anteroposterior and cross-table
lateral radiographs) were diagnostic of necrotizing enterocolitis, were
compatible with but not diagnostic of necrotizing enterocolitis, showed no
evidence of necrotizing enterocolitis, or showed pneumoperitoneum. All 10
reviewers agreed on only one of 17 examinations
[9]. For eight (47%)
examinations, half of the reviewers thought necrotizing enterocolitis was
present whereas the other half interpreted the films as normal
[9]. The results were not
evaluated based on the specialty practice area of the study participants
[9]. Di Napoli et al.
[8] also assessed interobserver
variability for detection of radiographic signs of necrotizing enterocolitis
in 297 radiographs taken of 57 neonates. That study included a large sample of
examinations (235/297) from neonates who did not have the diagnosis of
necrotizing enterocolitis. Kappa values for overall agreement ranged from 0.10
to 0.55 regarding detection of specific radiographic signs and the diagnosis
of necrotizing enterocolitis
[8].
Several possible explanations might account for the better interobserver
agreement in our study as compared with those of Mata and Rosengart
[9] and Di Napoli et al.
[8]. First, unlike the prior
two investigations, our study design included a series of reference
radiographs to standardize findings that constitute each score from 0 to 10.
Reviewing these examples may have improved interobserver agreement because
some of the inherent variability in nonspecific descriptive terms was thus
avoided. For example, the distinction between "mild" and
"moderate" bowel distention is subjective because there is no
absolute measure of normal, mild, or moderately dilated bowel
[1,
19,
27]. Our study participants
had sample examinations depicting mild versus moderate bowel distention.
Likewise, "featureless loops," "focal moderate
distention" bowel dilation, and other abnormal findings were categorized
before the study radiographs were interpreted. Second, to simulate our current
clinical practice, our investigation used a PACS workstation, which has
advantages such as the ability to magnify and adjust for overexposed or
underexposed examinations. Finally, the DAAS was used clinically by three
reviewers for at least 18 months and by one reviewer for approximately 9
months before participation in this study. In our clinical practice, previous
films and their accompanying reports are readily accessible on PACS and are
frequently reviewed when interpreting subsequent examinations. There fore,
before this investigation our study participants may have consciously or
unconsciously scored clinical cases to reach agreement with other readers. In
addition, all observers in our study were fellowship-trained pediatric
radiologists.
Our investigation has several limitations. First, because only two-view
abdominal radiographic series were included, we did not assess intra- and
interobserver agreement when only a single radiograph was available. At our
institution a standard examination for evaluation of a neonate or infant with
abdominal symptoms includes both an anteroposterior and a cross-table lateral
radiograph. We excluded single anteroposterior abdominal examinations because
these are generally obtained at our institution only for line placement or to
check other support apparatus. A further limitation of this study is that we
did not define objective measurements for each of the categories in our study.
For example, we did not define a measurement for bowel wall diameter that
would qualify as a distended loop of bowel. We considered defining such
measurements but decided that measurements would often not be reproducible,
especially in the setting of multiple overlapping bowel loops, and would
likely not easily translate into clinical practice.
A limitation of our scale is that we did not include "gasless
abdomen" as a category. A patient without bowel gas does not fit readily
into any of the categories in our scale in the absence of portal venous gas or
pneumoperitoneum. Furthermore, the radiographic presence of peri toneal fluid
was not included in our scale. However, radiography is insensitive for the
detection of ascites as compared with sono graphy, particularly when small
amounts are present [19]. In
clinical practice at our institution, sonography is performed when the
clinical question is whether ascites is present.
This investigation did not assess correlation of clinical outcomes with the
DAAS scoring system. Although the usefulness of the DAAS score will ultimately
hinge on its success in this arena, the first step in validating a scoring
system is to determine its reproducibility in clinical practice. The
usefulness of this scoring system in terms of predicting which neonates and
infants are at risk for developing necrotizing enterocolitis and its
life-threatening complications is currently under investigation at our
institution.
In conclusion, through this single-institution investigation, we have shown
substantial intra- and interobserver agreement in the interpretation of
abdominal radiographs of neonates and infants suspected of having necrotizing
enterocolitis when using a 10-point scale of abnormal findings. The DAAS
facilitates communication of abnormal findings between the radiologist and the
clinical services caring for the infant by standardizing level of concern and
degree of certainty that a patient has necrotizing enterocolitis.
Multiinstitutional investigations are needed to evaluate the feasibility of
translating this scale to other institutions. It is our hope that this scale
will be a valuable tool in improving communication with our clinical
colleagues.
As an aside, we continue to use this scoring system and it has been deemed
extremely useful by our neonatologists from the communication perspective.
Although correlation of this scoring system with clinical outcomes has not yet
been validated, our neonatologists use the DAAS score as part of their overall
clinical assessment. In clinical practice the descriptors (Fig.
1A,
1B,
1C,
1D,
1E,
1F,
1G,
1H,
1I,
1J,
1K,
1L,
1M,
1N,
1O,
1P,
1Q,
1R,
1S,
1T,
1U,
1V) are posted and readily
available for reference both to the radiologists who are interpreting the
films and to the clinicians who are caring for the neonates and infants. The
use of this scale may also facilitate research into the correlation between
radiographic findings and clinical outcomes.
References
- Guthrie SO, Gordon PV, Thomas V, Thorp JA, Peabody J, Clark RH.
Necrotizing enterocolitis among neonates in the United States. J
Perinatol 2003; 23:278
–285[CrossRef][Medline]
- Llanos AR, Moss ME, Pinzon MC, Dye T, Sinkin RA, Kendig JW.
Epidemiology of neonatal necrotizing enterocolitis: a population-based study.
Pediatr Perinat Epidemiol 2002;16
: 342–349[CrossRef][Medline]
- Sobel JL, Pearson ML, Gross K, et al. Information content and
clarity of radiologists' reports for chest radiography. Acad
Radiol 1996; 3:709
–717[CrossRef][Medline]
- Khorasani R, Bates DW, Teeger S, Rothschild JM, Adams DF, Seltzer
SE. Is terminology used effectively to convey diagnostic certainty in
radiology reports? Acad Radiol 2003;10
: 685–688[CrossRef][Medline]
- Patel NH, Lauber PR. The meaning of a nonspecific abdominal gas
pattern. Acad Radiol 1995;2
: 667–669[CrossRef][Medline]
- D'Orsi C. The American College of Radiology mammography lexicon: an
initial attempt to standardize terminology. AJR1996; 166:779
–780[Free Full Text]
- Berg WA, D'Orsi CJ, Jackson VP, et al. Does training in the Breast
Imaging Reporting and Data System (BI-RADS) improve biopsy recommendations or
feature analysis agreement with experienced breast imagers at mammography?
Radiology 2002;224
: 871–880[Abstract/Free Full Text]
- Di Napoli A, Di Lallo D, Perucci CA, et al. Interobserver
reliability of radiological signs of necrotising enterocolitis in a population
of high-risk newborns. Paediatr Perinat Epidemiol2004; 18:80
–87[CrossRef][Medline]
- Mata AG, Rosengart RM. Interobserver variability in the
radiographic diagnosis of necrotizing enterocolitis.
Pediatrics 1980;66
: 68–71[Abstract/Free Full Text]
- Rehan VK, Seshia MMK, Johnston B, Reed M, Wilmot D, Cook V.
Observer variability in interpretation of abdominal radiographs of infants
with suspected necrotizing enterocolitis. Clin Pediatr1999; 38:637
–643[Abstract/Free Full Text]
- Cichetti DV. A computer program for determining the significance of
the difference between pairs of independently derived values of kappa or
weighted kappa. Educ Psychol Meas 1981;41
: 189–193[Abstract]
- Landis JR, Koch GG. The measurement of observer agreement for
categorical data. Biometrics 1977;33
: 159–174[CrossRef][Medline]
- Faingold R, Daneman A, Tomlinson G, et al. Necrotizing
enterocolitis: assessment of bowel viability with color Doppler US.
Radiology 2005;235
: 587–594[Abstract/Free Full Text]
- Maalouf EF, Fagbemi A, Duggan PJ, et al. Magnetic resonance imaging
of intestinal necrosis in preterm infants. Pediatrics2000; 105:510
–514[Abstract/Free Full Text]
- Kao SC, Smith W, Franken EA Jr, Sato Y, Sullivan JH, McGee JA.
Contrast enema diagnosis of necrotizing enterocolitis. Pediatr
Radiol 1992; 22:115
–117[CrossRef][Medline]
- Patton WL, Willmann JK, Lutz AM, Rencken IO, Gooding CA. Worsening
enterocolitis in neonates: diagnosis by CT examination of urine after enteral
administration of iohexol. Pediatr Radiol1999; 29:95
–99[CrossRef][Medline]
- Epelman M, Daneman A, Navarro OM, et al. Necrotizing enterocolitis:
review of state-of-theart imaging findings with pathologic correlation.
RadioGraphics 2007;27
: 285–305[Abstract/Free Full Text]
- Merritt CR, Goldsmith JP, Sharp MJ. Sonographic detection of portal
venous gas in infants with necrotizing enterocolitis.
AJR 1984; 143:1059
–1062[Abstract/Free Full Text]
- Buonomo C. The radiology of necrotizing enterocolitis.
Radiol Clin North Am 1999;37
:1187
–1198[CrossRef][Medline]
- Kosloske A. Indications for operation in necrotizing enterocolitis
revisited. J Pediatr Surg 1994;29
: 663–666[CrossRef][Medline]
- Tam AL, Camberos A, Applebaum H. Surgical decision making in
necrotizing enterocolitis and focal intestinal perforation: predictive value
of radiographic findings. J Pediatr Surg2002; 37:1688
–1691[CrossRef][Medline]
- Frey EE, Smith W, Franken EA, Wintermeyer KA. Analysis of bowel
perforation in necrotizing enterocolitis. Pediatr
Radiol 1987; 17:380
–382[CrossRef][Medline]
- Daneman A, Woodward S, de Silva M. The radiology of neonatal
necrotizing enterocolitis (NEC): a review of 47 cases and the literature.
Pediatr Radiol 1978;7
: 70–77[CrossRef][Medline]
- Rabinowitz JG, Siegle RL. Changing clinical and roentgenographic
patterns of necrotizing enterocolitis. AJR1976; 126:560
–566[Abstract]
- Wexler HA. The persistent loop sign in neonatal necrotizing
enterocolitis: a new indication for surgical intervention?
Radiology 1978;126
: 201–204[Abstract]
- Berdon WE, Grossman H, Baker DH, Mizrahi A, Barlow O, Blanc WA.
Necrotizing enterocolitis in the premature infant. AJR1964; 83:879
–887
- Morrison SC, Jacobson JM. The radiology of necrotizing
enterocolitis. Clin Perinatol 1994;21
: 347–363[Medline]
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
Top
Abstract
Introduction
