Meta-analysis of the diagnostic performance of stress perfusion cardiovascular magnetic resonance for detection of coronary artery disease

Aim Evaluation of the diagnostic accuracy of stress perfusion cardiovascular magnetic resonance for the diagnosis of significant obstructive coronary artery disease (CAD) through meta-analysis of the available data. Methodology Original articles in any language published before July 2009 were selected from available databases (MEDLINE, Cochrane Library and BioMedCentral) using the combined search terms of magnetic resonance, perfusion, and coronary angiography; with the exploded term coronary artery disease. Statistical analysis was only performed on studies that: (1) used a [greater than or equal to] 1.5 Tesla MR scanner; (2) employed invasive coronary angiography as the reference standard for diagnosing significant obstructive CAD, defined as a [greater than or equal to] 50% diameter stenosis; and (3) provided sufficient data to permit analysis. Results From the 263 citations identified, 55 relevant original articles were selected. Only 35 fulfilled all of the inclusion criteria, and of these 26 presented data on patient-based analysis. The overall patient-based analysis demonstrated a sensitivity of 89% (95% CI: 88-91%), and a specificity of 80% (95% CI: 78-83%). Adenosine stress perfusion CMR had better sensitivity than with dipyridamole (90% (88-92%) versus 86% (80-90%), P = 0.022), and a tendency to a better specificity (81% (78-84%) versus 77% (71-82%), P = 0.065). Conclusion Stress perfusion CMR is highly sensitive for detection of CAD but its specificity remains moderate.


Introduction
Perfusion cardiovascular magnetic resonance (CMR) is an emerging technique for the detection of coronary artery disease (CAD). The technique is attractive because of its non-invasive nature and safe characteristics, and might potentially play a major role in future diagnosis and risk stratification guidelines for patients with suspected CAD. Several small studies have evaluated the diagnostic performance of stress perfusion CMR and some of those have been included in a previous meta-analysis [1]. In the current study we provide a comprehensive and contemporary meta-analysis of its diagnostic accuracy compared with an invasive coronary angiography (CA) used as a reference standard.

Search strategy
Using the combined medical subject headings (MeSH) of magnetic resonance, perfusion, and coronary angiography, with the exploded terms coronary artery disease; the MEDLINE, Cochrane Library and BioMedCentral databases were searched independently by two investigators (MH, GF) for all publications, in any language, before July 2009. In addition, the published reference lists of these articles were systematically searched.

Study eligibility
The search results were collated by the same two investigators (MH, GF), and duplicate or overlapping papers removed. Studies were eligible if: [1] stress perfusion CMR was used as a diagnostic test for significant obstructive CAD; [2] conventional invasive CA was used as the reference standard for diagnosing significant obstructive CAD, defined as a ≥50% diameter stenosis; and [3] the absolute numbers of true positive (TP), false positive (FP), true negative (TN), and false negative (FN) were reported, or could be derived. Studies were excluded if they were performed with a 0.5 or 1 Tesla MR scanner, if they included less than 10 patients, and if only abstracts from scientific meetings were published as the data provided may either be not sufficiently detailed or finalized. Any disagreements on eligibility were resolved by discussion and consensus between the two investigators.

Data extraction and quality assessment
Data extraction was performed independently by the two investigators (MH, GF) for each study. The following fields were recorded: study population size; gender distribution; mean age and standard deviation; number of patients with documented CAD; prevalence of CAD; relative timing of the two imaging procedures; the degree of blinding in interpretation of test results (both to the patient's clinical context and the results of the other imaging modality); type and brand of MR machine used; the type of perfusion stressor (adenosine, nicorandil, dipyridamole), and the number of side effects; the dose and injection rate of Gadolinium administrated; and the modality of MR image analysis (visual, or semi-quantitative). Any discrepancies were resolved by discussion and consensus between the two investigators. Where available, data was recorded separately at the level of coronary territories and coronary arteries. The study quality conformed to the Quality Assessment of Studies of Diagnostic Accuracy included in Systematic Reviews guidelines [2]. In one study, for which patients were evaluated both with 1.5 and 3T CMR, we used 1.5 T data in the metaanalysis. For the studies where analysis was performed with both 50% and 70% coronary stenosis definitions, we included results with the 70% definition in the pooled reported sensitivity and specificity.

Data synthesis and statistical analysis
Data analysis was performed at the level of the patient, the coronary territory and the coronary artery. Sensitivity and specificity were calculated using the TP, TN, FP, and FN rates [3,4]. From these were calculated the likelihood ratios, which express how much the odds of significant obstructive CAD change in the presence of either an abnormal stress perfusion CMR (positive likelihood ratio: PLR = sensitivity/(1-specificity)), or a normal stress perfusion CMR (negative likelihood ratio: NLR = (1-sensitivity)/specificity). Finally, the ratio of the PLR to the NLR was used to calculate the diagnostic odds ratio (DOR), which estimates how much greater the odds of having significant obstructive CAD are for patients with a positive test result compared with a negative one.
All these measures of diagnostic accuracy were calculated for each individual study and reported as point estimates with 95% confidence intervals. They were then combined using a random-effects model and each point estimate weighted by the inverse of the sum of its variance and the between-study variance. We also assessed between-study statistical heterogeneity using the Cochran Q chi-square tests (cut off for statistical significance P ≤ .10). Since diagnostic parameters are, by definition, interdependent, independent weighting may sometimes give spurious results and provide biased estimates; to overcome the interdependence problem, we computed the weighted symmetric summary receiver operating characteristic curve, with pertinent areas under the curve, using the Moses-Shapiro-Littenberg method [5][6][7]. All statistical calculations were performed with SPSS 14.0 (SPSS, Chicago, IL) and Meta-DiSc [8], and significance testing was at the two-tailed 0.05 level [9].

Results
Database and literature searches retrieved 263 citations, amongst which 55 relevant publications were identified ( Figure 1). Further scrutiny led 20 papers to be rejected either because of overlapping data, or exclusion criteria were met (employed 0.5 or 1 T CMR, or inclusion criteria were absent (impossible to find or calculate absolute figures from presented data). Therefore, 35 studies were finally included in the meta-analysis , all of which had been published between 2000 and 2009. Study and population characteristics are summarized in Table 1, and the results of the pooled analyses are summarized in Table 2. Dose of contrast Gadolinium administrated range from 0.025 to 0.15 mmol/kg, with an injection rate varying from 3 to 10 mL/s. Quality assessments for all included studies are shown in Table 3. The 35 papers eligible for the analyses comprised 2,456 patients, and of the 2,154 patients for whom gender and the age were speci-   fied, 1,481 were males (68.7%) and the mean age was 61.3 years.
Per-artery analysis pooled 8 datasets and demonstrated for left anterior descending artery (LAD), circumflex artery (CX) and right coronary artery (RCA), respectively, sensitivities of 83%, 76% and 78% and specificities of 83%, 87%, and 87%. Statistical heterogeneity was  Item 1: was the spectrum of patients representative of the patients who will receive the test in practice?; Item 2: were selection criteria clearly described?; Item 3: is the reference standard likely to correctly classify the target condition?; Item 4: is the time period between reference and standard and index test short enough to be reasonably sure that the target condition did not change between the two tests?; Item 5: did the whole sample or a random selection of the sample, receive verification using a reference standard of diagnosis?; Item 6: did patients receive the same reference standard regardless of the index test results?; Item 7: was the reference standard independent of the index test (i.e. the index test did not form part of the reference standard); Item 8: was the execution of the index test described in the sufficient detail to permit replication of the test; Item 9: was the execution of the reference standard described in the sufficient detail to permit its replication?; observed for all the performance measurements except sensitivity and negative likelihood ratio for LAD and CX, and diagnostic odds ratio for CX.

Discussion
This meta-analysis showed stress perfusion CMR to have a high sensitivity (89%) and a moderate specificity (80%) at patient level for the diagnosis of significant obstructive CAD in patients with high prevalence of CAD (57%). We included twelve more studies (on stress perfusion CMR) than the previous meta-analysis by Nandalur et al. [1], which showed a similar diagnostic performance with a pooled sensitivity and specificity of respectively 90% and 81% from 14 perfusion studies. A high false positive rate could have driven the relatively low specificity, and may be due to perfusion defects caused by: [1] dark rim artefacts, the hypo-intensities along the endocardial border of the left ventricular myocardium seen during first-pass transit of a MR contrast medium, thought to be due to a combination of the gadolinium bolus, motion and resolution [45]; [2] the presence of microvascular disease; and [3] spontaneous or therapeutic re-opening of a coronary artery supplying an area of myocardial infarction that has persistent microvascular obstruction [28,32]. Alternatively, because CA detects luminal morphology rather than the functional significance of a stenosis, a false positive CMR results may in fact represent a 'false negative' angiogram in the context of angiographically 'invisible' small vessel disease capable of inducing subendocardial ischemia [40]. This potential source of error could be minimised if the hemodynamic significance of an epicardial coronary artery stenosis were to be determined by the measurement of the fractional flow reserve (FFR) during CA. If validated, this may represent a better reference standard than CA alone. However, although three studies found there to be a good correlation between the performance of stress perfusion CMR and CA with FFR measurement [31,33,35], sufficient data was not present to evaluate its accuracy in this study.
Another point to outline is that for some studies [11,17,19], different decision thresholds to diagnose perfusion CMR as abnormal were appraised: for these studies, the reported sensitivity and specificity could be considered as optimistic because the end points was chosen retrospectively.
In addition, there was a large range of contrast doses used in the individual studies, with the dose of gadolinium administered in the included studies varying by 6fold, with dose ranging from 0.025 to 0.15 mmole/kg. Although currently there is no consensus regarding the optimal dose and injection rates for perfusion CMR, two multicenter dose-ranging studies have evaluated the impact of contrast dose on the performance of perfusion CMR using a visual analysis [46,47]. In the first, Wolff et al. considered a low dose of 0.05 mmol/kg to be at least as efficacious as any higher dose, and hypothesized that higher doses preformed less well because of the increased likelihood and intensity of artefacts at these doses [46]. However, in the MR-Impact study, Schwitter et al. found better results were obtained using 0.1 mmol/kg [47].
The fact that the meta-analysis demonstrated a low NLR for stress perfusion CMR suggests that a negative test result may in fact be more clinically useful. This is in keeping with several reports, in different clinical settings, of improved prognosis associated with a normal adenosine stress perfusion CMR scan [48][49][50]. This meta-analy-sis also demonstrated adenosine to be superior to dipyridamole as the vasodilating stressor agent. Adenosine may also be safer, with minor side effects of flushing and headache being reported to occur more frequently that any severe adverse effects [51]. Its shorter half life (< 10 s) is an added advantage. Moreover, adenosine has documented safety in the context of non-ST elevation acute coronary syndromes (in a study of 72 patients only one demonstrated intolerance), and in recent ST elevation myocardial infarction [14,22,34].

1-specificity
From this analysis, visual assessment of stress perfusion CMR provided a higher sensitivity but a lower specificity than semi-quantitative assessment. Currently there is no consensus on the superiority of visual over semi-quantitative assessment, or on which method of semi-quantitative assessment should be used. However, the drawbacks of semi-quantitative assessment are that it is more timeconsuming, hence not ideal for day-to-day clinical purposes, and the lack of any homogeneous post-processing protocols. Therefore, visual assessment is currently the method most often used in routine clinical practice.
Only 4 studies were performed using 3T CMR, which provides improved resolution [32,38,39,43]. Enhanced sensitivity has been reported [32] and attributed to the higher signal-to-noise and contrast-to-noise ratios permitting improved detection of endocardial perfusion defects. Although most authors argue that the increased prevalence of dark rim artefacts at these higher field strengths (ranging from 8 up to 82%) does not hamper myocardial perfusion analysis [32,39,43], Gebker disagrees and suggests they could limit specificity by increasing false positive rates [38]. In this analysis, 3T CMR was also found to have a decreased specificity, indicating that higher false positive rates may be a real problem. Further studies will be necessary if this controversy is to be resolved.
The results of the per-territory-based analysis showed the anticipated decrease in sensitivity and increase in specificity seen when moving from the level of the patient to that of the coronary territory. Among the 8 studies that performed a coronary-artery level analysis, stress perfusion CMR had a higher sensitivity for detection of significant coronary disease in the LAD artery, compared with the CX and RCA. A possible explanation for this finding may have been the use of a surface radiofrequency coil, which led to lower signal intensities in the more distant inferior and lateral segments.

Study limitations
Although conventional CA is the established technique for diagnosing significant CAD in routine clinical practice, it remains an imperfect reference standard due to its inability to evaluate the hemodynamic significance of a stenosis.
Substantial inter-study heterogeneity in multiple performance characteristics were observed. Therefore, the pooled performance indices and their interpretation have to be treated with a degree of caution, even though the random-effects model used throughout the analysis should have compensated for this. The observed heterogeneity may have been due to variations in: (i) the image acquisition technique (MR scanner manufacturer, 1.5T or 3T field strengths, pulse sequence, number of slices, contrast dose and rate of infusion); (ii) the interpretation method (visual or semi-quantitative, post-processing techniques); (iii) the patient selection criteria (exclusion or inclusion of patients with prior myocardial infarction, patient populations with differing prevalence of CAD); and (iv) in the definition of significant obstructive CAD (50% or 70%). We noticed, as expected, that studies which performed analysis for 50% and for 70% coronary artery stenosis thresholds, reported an increased sensitivity and a decreased specificity when moving thresholds from 50% to 70% [29,33,36].
These general limitations of stress perfusion CMR could be addressed in future multi-centre studies if standardized imaging protocols, post-processing techniques and patient selection criteria are employed.

Conclusion
Stress Perfusion CMR has a high sensitivity and moderate specificity for the diagnosis of significant obstructive CAD compared with CA in patients with a high prevalence of the disease.
Future technical developments that increase spatial and temporal resolution whilst reducing artefacts may further improve the diagnostic performance of stress perfusion CMR, and in particular improve its specificity [32]. Currently, however, the low NLR makes stress perfusion CMR particularly accurate and useful in ruling out significant CAD.