| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
1 Russell H. Morgan Department of Radiology and Radiological Science, Johns
Hopkins University School of Medicine, 600 N Wolfe St., Central Radiology
Viewing Area, Rm. 117, Baltimore, MD 21287.
2 Department of Medicine, Johns Hopkins University School of Medicine,
Baltimore, MD 21287.
3 Department of Health Policy and Management, Johns Hopkins University Bloomberg
School of Public Health, 615 N Wolfe St., Baltimore, MD 21205.
Received January 20, 2004;
accepted after revision May 10, 2004.
Conducted by the Johns Hopkins Evidence-Based Practice Center and supported
by the Agency for Healthcare Research and Quality, United States Department of
Health and Human Services, Rockville, MD, through contract 290-97-0006. The
authors are responsible for the content of this article, including any
treatment recommendations. No statement in this article should be construed as
an official position of the Agency for Healthcare Research and Quality or of
the United States Department of Health and Human Services.
Abstract
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
MATERIALS AND METHODS. We selected all systematic reviews published before December 2003 that evaluated the accuracy of CT angiography for the diagnosis of pulmonary embolism. We also selected all prospective studies from the same time period in the primary literature in which all subjects underwent both CT and conventional angiography, the latter being considered the reference standard. Articles were identified through a computerized MEDLINE search and by other means. The quality and content of each article were evaluated independently by pairs of researchers.
RESULTS. Six systematic reviews and eight primary studies were selected. The combined sensitivities of CT for detecting pulmonary embolism ranged from 66% to 93% across the systematic reviews and the combined specificities ranged from 89% to 97%. Only one of the reviews reported a combined sensitivity of greater than 90%. Among the eight primary studies, the sensitivities ranged from 45% to 100% and specificities ranged from 78% to 100%. Only three of the eight primary studies reported a sensitivity greater than 90%. None of the primary studies used scanners with four or more detectors.
CONCLUSION. A systematic literature review revealed a wide range of reported sensitivities, only a minority of which exceeded 90%. Pooled estimates of sensitivity and specificity reported by systematic literature reviews should be interpreted with caution because of potential selection bias and heterogeneity in the reviewed studies. Accuracy studies of recent generations of MDCT scanners are not yet available despite the current dissemination of this technology.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
With the advent of high-speed helical scanners in the early 1990s, it became possible to examine the pulmonary arteries for emboli using CT [2]. The advantages of helical CT include rapid examination time, widespread availability in emergency clinical settings, safety because of noninvasiveness, low cost compared with conventional pulmonary arteriography, and the concurrent examination of the lung parenchyma. Helical scanners have since become widely available, and examination of the pulmonary arteries on helical CT has become a routine practice [3]. The recent generation of MDCT scanners provides increasingly detailed images of the pulmonary vasculature [4] and presumably have even greater diagnostic accuracy. Given the high reported accuracy of helical CT, it is reasonable to consider whether that technique can replace traditional imaging techniques for detecting pulmonary embolism (i.e., ventilation–perfusion scintigraphy and pulmonary arteriography by catheterization).
Despite the increasing use of contrast-enhanced CT for the detection of pulmonary embolism, the reported accuracy of this test varies among published reports [5], with some of the most recent studies reporting sensitivities of approximately 50% [6, 7]. Because accuracy is a key consideration in the decision to use diagnostic tests, we conducted a systematic literature review to summarize the best available evidence concerning the accuracy of CT angiography for the diagnosis of pulmonary embolism. Our examination of the literature included stringent methodologic criteria for selecting articles to review that we expected would improve the reliability of our summary of the evidence regarding the accuracy of CT compared with the accuracy reported in previous reviews [8–11]. In our systematic review of the literature, we focused on sensitivity and specificity as the indicators of diagnostic accuracy, because these indicators are most commonly reported in the literature.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Literature Search
To identify systematic reviews to include in part 1 of this review, we
performed a computerized search of the MEDLINE database through the PubMed
service of the National Library of Medicine in Bethesda, MD, for review
articles in the English language that pertained to the main search terms
"pulmonary embolism" and "computed tomography." We
used the following search phrase in which "[pt]" represents
"publication type": (quantitative* OR methodolog* OR systematic*
OR meta-analysis OR "metaanalysis" OR "meta analysis"
OR "meta-analyses" OR "metaanalyses" OR "meta
analyses" OR (MEDLINE AND review [pt]) OR "clinical
conference"[pt] OR "consensus development conference" [pt]
OR "guideline" [pt] OR "meta analysis" [pt] OR
"practice guideline"[pt] OR (review [pt] AND systematic*)) AND
(pulmonary embolism) AND (computed tomography).
To identify original articles for inclusion in part 2 of this review, we performed a MEDLINE search with the following phrase: evaluation AND (pulmonary embolism) AND (computed tomography). Both searches included articles published between 1966 and December 2003. Limiting the searches to the English language was done primarily for practical reasons, but there is some evidence that the resulting language bias is small [12].
To ensure a comprehensive search of the literature, we supplemented the computerized MEDLINE search in several ways. The search phrase was modified and used to search the Cochrane Database of Systematic Reviews (Update Software) and the Micromedex database (Micromedex) for relevant articles. The tables of contents of the six most commonly cited clinical imaging journals (Radiology, Journal of Nuclear Medicine, Magnetic Resonance in Medicine, Seminars in Nuclear Medicine, American Journal of Roentgenology, and Journal of Computer Assisted Tomography) were reviewed to identify additional articles. These six journals were selected from a list of clinical imaging journals identified and indexed by the Radiological Society of North America [13]. Frequency of citation was obtained from the Science Citation Index (Thomson ISI). After developing our final list of eligible articles, we queried an external panel of experts in the field for relevant articles missing from the final list. Finally, the reference lists of all retrieved articles were examined to identify additional articles.
Article Selection
For part 1 of this review, we selected all systematic literature reviews of
CT angiography for the diagnosis of pulmonary embolism. To be included, the
article had to report sensitivity and specificity of CT angiography. The
article also had to be a systematic review, defined as one in which the
articles were selected according to specific inclusion criteria
[14].
For part 2 of this review, we selected all prospective studies in which the accuracy of CT angiography for the detection of pulmonary embolism was measured against conventional pulmonary arteriography as the reference standard. We included only studies of diagnostic test accuracy [15] in which all subjects uniformly underwent both CT and conventional angiography, the latter test being the reference standard. This criterion was designed to avoid selecting studies that included a workup bias [16]. Articles not reporting original data were excluded. Studies evaluating electron beam CT were excluded because this technology is not routinely available. Also excluded were case reports and conference abstracts not associated with a full research article. For the purposes of this review, we defined MDCT as an examination involving four or more detectors.
Starting with all citations returned by the literature search, we used the following protocol to identify articles meeting our selection criteria: The abstract of each citation was reviewed by two members of our multidisciplinary study team working independently. Each abstract reviewer indicated whether the article met the inclusion criteria for either of the two parts of our review. If there was initial disagreement regarding article eligibility, the two abstract reviewers met to arrive at a consensus. If either of the abstract reviewers was unable to determine article eligibility from the abstract, the full article was retrieved and evaluated by each reviewer independently to determine its eligibility.
Quality Evaluation and Data Extraction
For part 1 of this review, we developed a questionnaire
[17] for evaluating the
quality of the eligible systematic reviews. This questionnaire contained 12
items and was based on several published systems for evaluating systematic
reviews
[18–23].
The items were divided into five categories, each evaluating a different
component of the systematic literature review process: search methods, study
inclusion criteria, assessment of the quality of included studies, combining
of results, and aims and conclusions.
For part 2 of this review, we developed an additional questionnaire [17] for evaluating the quality of the eligible primary articles. This questionnaire contained 18 items and was derived from those used in previous projects evaluating diagnostic testing [24, 25]. The items in this questionnaire were divided into five categories: representativeness of the study population, sources of potential bias and confounding, description of the testing protocols, test interpretation methods, and statistical quality and interpretation.
The full text of all eligible articles was retrieved and independently evaluated by two members of the study team with one of the two quality evaluation instruments. Each item on either questionnaire was given a score of 0 (item not satisfied), 1 (item partially satisfied), or 2 (item fully satisfied). Any disagreements in scoring were resolved through consensus in a meeting of the two team members.
Data relevant to this review were recorded on data extraction forms by two team members independently. Relevant data included demographic information, study aims, inclusion criteria, reported results, and stated conclusions. The main outcomes recorded for both parts of this review were sensitivity and specificity for the detection of pulmonary embolism. The two team members met to form a consensus concerning all data for which there was initial discrepancy. Separate data extraction forms were used for each of the two parts of our review. Author or journal names were not masked for reviewers during the quality evaluation or data extraction processes because previous work has shown that masking is unlikely to affect the results of data abstraction [26].
If appropriate, proportions were reported along with the exact binomial 95% confidence intervals. Summary-weighted means of proportions were calculated by adding the numerators and denominators of the individual proportions and reporting the ratio between these two sums.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Review characteristics.—Six systematic reviews have examined the use of helical CT for the diagnosis of pulmonary embolism [5, 8–11, 27] (Table 1). The most recent included the literature published before December 2000 [11]. A major difference in these systematic reviews was the reference standard against which CT was compared. Two of the reviews examined only studies in which the reference standard was pulmonary arteriography [5, 27]. Two reviews defined the reference standard as either pulmonary arteriography or ventilation–perfusion scanning [10, 11]. The remaining two reviews did not limit the reference standard to specific imaging techniques [8, 9]. Two of the reviews included an article evaluating contrast-enhanced electron beam CT [5, 27].
|
Quality.—Table 1 summarizes our assessment of the quality of the systematic reviews. The overall quality scores for all the reviews ranged from 55% to 78%. Overall, the statements of study aims and conclusions received the highest quality scores, whereas the descriptions of the search methods received the lowest quality scores.
Findings.—The findings of the systematic reviews are shown in Table 2. All the reviews reported sensitivity and specificity as the main indices of test performance for helical CT in diagnosing pulmonary embolism. In five reviews, the sensitivities and the specificities of each reviewed study were averaged, with weighting according to the sample size of each study. The combined sensitivities of CT across reviews ranged from 66% to 93%, and the combined specificities ranged from 89% to 98%. Only one review reported a combined sensitivity of greater than 90%. In one of the reviews, the combined sensitivity and the combined specificity were not reported because the authors thought that the heterogeneity of studies included did not allow mathematic combination [8]. In that review, sensitivity was reported as ranging from 53% to 100%, and specificity was reported as ranging from 81% to 100%.
|
Part 2: Systematic Review of Primary Literature
Selection.—The literature search identified 138 citations
for consideration. Review of the abstracts for these citations resulted in the
identification of 22 primary articles. After retrieving the full text of these
articles, the reviewers excluded 14 of the studies described, seven because
they were not prospective studies of diagnostic test accuracy
[15] and seven because they
did not meet other inclusion criteria. The remaining eight primary articles
were selected for our review.
Study characteristics.—Table 3 summarizes key aspects of the eight eligible primary studies of CT angiography, which were published between 1992 and 2001 [2, 6, 7, 28–32]. All studies were diagnostic test evaluations in which all participants underwent both CT and conventional angiography, the latter being the reference standard. None were multicenter studies, and none of the reports stated the specific dates of participant recruitment. Although some of these primary studies appeared in the systematic reviews in part 1, none of the systematic reviews included all of the primary studies that we selected for our primary literature review. Furthermore, the systematic reviews included a number of articles that were excluded from our primary literature review.
|
Only one study used explicit clinical findings to define the suspicion of pulmonary embolism [7]. In six of the studies, clinical suspicion of pulmonary embolism was implied because all participants in these studies were referred for imaging [2, 6, 28–30, 32]. In one study, it was unclear whether patients were enrolled because of referral for imaging or because of explicit symptomatologic criteria [31].
The studies used a variety of image acquisition protocols (Table 4). None used an MDCT scanner (four or more detectors). Except for one study of a dual-detector scanner [32], all studies used a conventional single-detector helical CT. For reference, the single-detector CT protocol currently used at our institution to evaluate for pulmonary embolism consists of a single breath-hold, section thickness of 3 mm, contrast material dose of 42 g of iodine, and contrast bolus duration of 40 sec. By comparison, the 4-MDCT protocol at our institution uses a section thickness of 1.25 mm.
|
Quality.—The study quality scores are given in Table 5. For the eight eligible primary studies, the scores ranged from 44% to 84%. The two categories with the lowest average quality scores across the eight primary studies were those describing the subjects included and addressing potential bias and confounding.
|
Findings.—The eight primary studies reported data on 443 individuals with the prevalence of pulmonary embolism ranging from 27% to 70%. The basic demographic characteristics of the participants in each of the studies are given in Table 3. The results of each study are summarized in Table 6. The reported sensitivity of CT angiography ranged from 45% to 100%, and the reported specificity ranged from 78% to 100%. Only three of the eight studies reported an overall sensitivity of greater than 90%, whereas six of the studies reported an overall specificity of greater than 90%. The only study reporting perfect accuracy was the one that enrolled patients with clinically suspected massive pulmonary embolism [28]; this study also had the highest prevalence of pulmonary embolism among the primary studies we reviewed.
|
To summarize the primary studies graphically, we plotted a representative true-positive rate (sensitivity) and false-positive rate (1 – specificity) for each study (Fig. 1). We specified that each representative sensitivity and specificity pair be calculated using data from all the participants in the corresponding study, using the cutoff that yielded the best test performance if several cutoffs were reported. The greater variability in sensitivities relative to the variability in specificities is also apparent in Figure 1. Figure 2 shows no apparent relationship between the prevalence of pulmonary embolism and the reported sensitivity and specificity.
|
|
As is evident from Table 4, a variety of image acquisition protocols have been reported in the literature, and this variety may be due to technical developments over time. However, considering the eight primary studies in chronologic order, we found no apparent trends in sensitivity or specificity over time (Fig. 3).
|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Almost all of the systematic reviews reported a summary sensitivity and specificity calculated as weighted means of the reviewed studies. Such calculations require the assumption that the reviewed studies are similar enough to be pooled, (i.e., each study has the same true underlying sensitivity and specificity so that random variation is the only source of variation among the results of different investigations) [33]. This assumption is probably not valid in the eight primary studies we reviewed, raising serious doubt about the validity of pooling sensitivity and specificity. For example, Figure 1 suggests that two of the studies are outliers, having sources of variance outside of random variation [6, 7]. The study by Velmahos et al. [7] reported the lowest sensitivity, but theirs was also the only study in which all participants came from a specific clinical setting, a surgical ICU, which might present unique diagnostic challenges not found in the general patient population [34, 35]. Poor interobserver agreement in the detection of subsegmental emboli could also be a source of variability. Therefore, the pooled sensitivity and specificity calculated by other reviews may actually have little value because of potential underlying heterogeneity. For this reason, we omitted calculation of the pooled sensitivity and specificity and did not attempt further summary of the data with more sophisticated meta-analysis.
Two primary studies suggested that the relatively low sensitivity may be related to whether image interpretation included the finding of subsegmental clots that were seen on the reference tests. Velmahos et al. [7] included interpretation of subsegmental clots, and their study was associated with the lowest sensitivity of all of the studies reviewed. In the study by Goodman et al. [29], inclusion of subsegmental clots lowered the sensitivity from 86% to 64%. However, the study by Qanadli et al. [32] differed from this pattern because it reported relatively high sensitivity and specificity despite the inclusion of subsegmental clot findings. Therefore, in the studies reviewed, we cannot establish a definite relation between test accuracy and the vessel level interpreted.
The sensitivity of CT angiography found in our examination of both the primary literature and systematic reviews was generally higher than that found in a large study of outpatients, which reported a sensitivity of 70% and a specificity of 91% [36]. The latter study incorporated other imaging techniques as well as clinical follow-up to establish the presumptive diagnosis of pulmonary embolism rather than relying on pulmonary arteriography alone. Use of multiple clinical methods for identifying cases of pulmonary embolism is likely to find cases that are not detectable on imaging, which lowers the apparent sensitivity of imaging compared with the sensitivities reported in the literature we reviewed.
We note some important limitations in the primary literature that we examined. First, participants in all but one of the studies [7] were enrolled because of a suspicion of pulmonary embolism that led to referral for imaging. This introduced a potential selection bias in the study populations because nothing is known about individuals who were not referred for diagnostic imaging but who may have pulmonary embolism (e.g., patients with clinically unsuspected pulmonary embolism or those with known deep venous thrombosis who are treated without further evaluation for respiratory symptoms). The real effect of this potential selection bias is difficult to determine from the data, however. Individuals referred for imaging may have been selected because of clinically obvious (rather than occult) disease. Perhaps such individuals have a form of disease that is easier to detect on imaging than the typical case (inflating sensitivity and specificity), as exemplified by a study that included only patients suspected of having massive pulmonary embolism [28]. On the other hand, physicians may have referred only clinically difficult cases that could have more subtle imaging findings than clinically obvious cases, potentially deflating sensitivity and specificity.
The published studies exhibited obvious heterogeneity in the prevalence of pulmonary embolism. Although disease prevalence strongly influences the positive and negative predictive values of a test, it theoretically should not affect the sensitivity and specificity of a test. However, if the variation in prevalence is indicative of a variation in disease spectrum or severity, then sensitivity and specificity may be affected. Again, this principle is exemplified by the study of patients believed to have massive pulmonary embolism [28].
Because of its traditional role as the reference standard in the available literature, conventional pulmonary arteriography was selected as the reference standard for diagnosis of pulmonary embolism in our systematic review. However, some evidence suggests that conventional arteriography may not be an adequate reference standard. When searching for small, subsegmental emboli on conventional arteriograms, interobserver agreement has been reported to be only 45–66% [34, 35]. If the diagnostic imperfections of conventional arteriography are statistically independent of those of CT, then the true sensitivity and specificity of CT may be higher than has been observed [37].
Some published evidence that we reviewed suggests that the sensitivity and negative predictive value of CT are not high enough to rule out pulmonary embolism on the basis of negative findings on a CT scan. This conclusion may be negated by more recent studies in which clinical outcome was considered the diagnostic reference [4]. These studies report a low incidence of a subsequent clinical diagnosis of pulmonary embolism after negative findings on a CT examination (i.e., a high negative predictive value). Clinical outcome is arguably a more appropriate end point than lesion (embolus) detection when evaluating a diagnostic CT test. As this line of evidence grows, a systematic review of this literature will be warranted.
The literature considered in this review represents approximately a decade of experience with helical CT in the diagnosis of pulmonary embolism. Although substantial technical improvements have occurred during this period, no clear corresponding improvement in sensitivity or specificity is apparent (Fig. 3). However, as with any systematic literature review, our results are limited by what has been published. The literature has apparently not been able to keep up with the continued rapid advances in CT technology. Despite the widespread dissemination of MDCT scanners, our review did not identify published accuracy studies of the performance of recent generations of MDCT scanners. We also found no primary accuracy studies published since 2001 and no systematic reviews since 2002. Available since 1999, MDCT scanners are capable of faster scanning times and significantly thinner image sections than single-detector models, resulting in substantially improved visualization of pulmonary vascular detail [4]. A slice thickness of 1–2 mm is now standard practice, which is less than half the average slice thickness reported in the published evidence we found. Therefore, substantial technical advances have occurred since the last published article. A multicenter clinical trial, the Prospective Investigation of Pulmonary Embolism Diagnosis II (PIOPED II) [38], is currently underway to assess the efficacy of MDCT in patients in which acute pulmonary embolism is suspected.
In summary, our examination of both systematic reviews and primary studies revealed a moderate amount of variation in the reported sensitivity of CT angiography for the diagnosis of pulmonary embolism, ranging from 45% to 100%, with only a minority reported above 90%. Reported specificity was generally greater than 90% with less variability. Sensitivity was not clearly related to certain study design characteristics (whether or not a prospective study design was used), prevalence of pulmonary embolism, or smallest arterial level (segmental or subsegmental) interpreted by the radiologists. MDCT represents a significant technical advance that allows visualizing finer pulmonary vascular detail and providing potentially greater diagnostic accuracy. Despite current dissemination of MDCT scanners, systematic evaluations of this technology are not yet available.
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
This article has been cited by other articles:
|
|
 |
|
  H. A. Ghofrani, M. W. Wilkins, and S. Rich Uncertainties in the Diagnosis and Treatment of Pulmonary Arterial Hypertension Circulation, September 9, 2008; 118(11): 1195 - 1201. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. Glassroth Imaging of Pulmonary Embolism: Too Much of a Good Thing? JAMA, December 19, 2007; 298(23): 2788 - 2789. [Full Text] [PDF]
|
|
|
|
 |
|
 S. Siragusa, A. Malato, F. Falaschi, F. Porro, R. Anastasio, A. Giarratano, L. L. Coco, M. C. Buonanno, E. Maggi, M. A. Bressan, et al. Deferral of assessment of pulmonary embolism Haematologica, March 1, 2007; 92(3): 410 - 413. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. B. Segal, J. Eng, L. J. Tamariz, and E. B. Bass Review of the Evidence on Diagnosis of Deep Venous Thrombosis and Pulmonary Embolism Ann. Fam. Med, January 1, 2007; 5(1): 63 - 73. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 V. D. Raptopoulos, P. B. Boiselle, N. Michailidis, J. Handwerker, A. Sabir, J. A. Edlow, I. Pedrosa, and J. B. Kruskal MDCT Angiography of Acute Chest Pain: Evaluation of ECG-Gated and Nongated Techniques Am. J. Roentgenol., June 1, 2006; 186(6_Supplement_2): S346 - S356. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. G. Ravenel, P. Costello, U. J. Schoepf, J. Eng, and J. B. Segal CT in the Diagnosis of Pulmonary Embolism Am. J. Roentgenol., May 1, 2005; 184(5): 1707 - 1708. [Full Text]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
