Skip to main content

Comparison of different methods for the estimation of aortic pulse wave velocity from 4D flow cardiovascular magnetic resonance

Abstract

Background

Arterial pulse wave velocity (PWV) is associated with increased mortality in aging and disease. Several studies have shown the accuracy of applanation tonometry carotid-femoral PWV (Cf-PWV) and the relevance of evaluating central aorta stiffness using 2D cardiovascular magnetic resonance (CMR) to estimate PWV, and aortic distensibility-derived PWV through the theoretical Bramwell-Hill model (BH-PWV). Our aim was to compare various methods of aortic PWV (aoPWV) estimation from 4D flow CMR, in terms of associations with age, Cf-PWV, BH-PWV and left ventricular (LV) mass-to-volume ratio while evaluating inter-observer reproducibility and robustness to temporal resolution.

Methods

We studied 47 healthy subjects (49.5 ± 18 years) who underwent Cf-PWV and CMR including aortic 4D flow CMR as well as 2D cine SSFP for BH-PWV and LV mass-to-volume ratio estimation. The aorta was semi-automatically segmented from 4D flow data, and mean velocity waveforms were estimated in 25 planes perpendicular to the aortic centerline. 4D flow CMR aoPWV was calculated: using velocity curves at two locations, namely ascending aorta (AAo) and distal descending aorta (DAo) aorta (S1, 2D-like strategy), or using all velocity curves along the entire aortic centreline (3D-like strategies) with iterative transit time (TT) estimates (S2) or a plane fitting of velocity curves systolic upslope (S3). For S1 and S2, TT was calculated using three approaches: cross-correlation (TTc), wavelets (TTw) and Fourier transforms (TTf). Intra-class correlation coefficients (ICC) and Bland-Altman biases (BA) were used to evaluate inter-observer reproducibility and effect of lower temporal resolution.

Results

4D flow CMR aoPWV estimates were significantly (p < 0.05) correlated to the CMR-independent Cf-PWV, BH-PWV, age and LV mass-to-volume ratio, with the strongest correlations for the 3D-like strategy using wavelets TT (S2-TTw) (R = 0.62, 0.65, 0.77 and 0.52, respectively, all p < 0.001). S2-TTw was also highly reproducible (ICC = 0.99, BA = 0.09 m/s) and robust to lower temporal resolution (ICC = 0.97, BA = 0.15 m/s).

Conclusions

Reproducible 4D flow CMR aoPWV estimates can be obtained using full 3D aortic coverage. Such 4D flow CMR stiffness measures were significantly associated with Cf-PWV, BH-PWV, age and LV mass-to-volume ratio, with a slight superiority of the 3D strategy using wavelets transit time (S2-TTw).

Background

Aortic stiffening is an early sign of remodeling and functional changes in arterial hemodynamics, and a marker of cardiovascular aging [1, 2]. The clinical usefulness of aortic stiffness has been previously demonstrated through its significant associations with adverse left ventricular (LV) remodeling, coronary heart disease, atherosclerosis and elevated mortality [3,4,5,6]. Aortic stiffness is commonly assessed using carotid-femoral (Cf) pulse wave velocity (PWV), which has been shown to be an accurate non-invasive alternative [7,8,9] to cardiac catheterization [10] in the in vivo measurement of pulse wave velocity (PWV). PWV is defined as the distance (D) travelled by the pressure wave between two anatomical locations, divided by the transit time (TT) spent by the wave to travel such distance.

Cardiovascular magnetic resonance imaging (CMR) offers excellent anatomical coverage and its anatomical and velocity-encoded sequences allow an accurate estimation of aortic geometry (length, diameters, volumes) as well as blood flow-derived indices in the thoracic aorta. In particular, two-dimensional through-plane phase-contrast CMR (2D phase contrast (PC)-CMR) has been used for the estimation of aortic (ao) PWV (aoPWV), using arch length from the ascending (AAo) to the descending (DAo) aorta, divided by the TT derived from a single acquisition plane positioned perpendicularly to both the AAo and DAo [11,12,13,14]. Alternatively, ascending aorta PWV estimation was also proposed using the theoretical Bramwell-Hill (BH) model and aortic distensibility, which is commonly derived from aortic cine CMR and central pulse pressure [15,16,17].

CMR with full three-dimensional anatomical coverage and velocity encoding in the three directions resolved throughout the cardiac cycle (4D flow CMR) has been developed, opening new and unique opportunities to both visualize and quantify cardiovascular complex blood flow [18, 19]. 4D flow CMR has several advantages, including its excellent 3D anatomical and velocity coverage which enables an accurate estimation of aortic arch length and aortic flow rates and velocities. Furthermore, as compared with 2D PC-based approaches, 4D flow PWV is better suited to diseases with complex arterial geometry and tortuosity or with heterogeneous stiffness patterns along the arterial tree that can be associated with atherosclerosis [20] or changes in arterial size [21, 22]. Moreover, the estimation of aortic PWV from 4D flow CMR [20] has been shown to be feasible using either TT [20, 23,24,25] or plane fitting [20] approaches. The present study aims to provide a comprehensive comparison of both TT and plane fitting-based methods for aortic PWV estimation from 4D flow CMR in healthy subjects, in terms of: 1) associations with the CMR-independent well-established Cf-PWV measure, 4D flow CMR-independent BH-PWV, age and LV mass-to-volume ratio, as well as 2) inter-observer reproducibility and robustness to temporal resolution.

Methods

Study population and data acquisition

We retrospectively studied 47 healthy subjects 20 to 79 years (49.5 ± 18 years, 23 males), without overt cardiovascular disease. Approval of the local Institutional Review Board and informed consent was obtained from all participants. All subjects had a CMR exam on a 3 T system (Discovery 750w GEM, General Electric Healthcare, Waukesha, Wisconsin, USA), including 4D flow acquisitions with retrospective electrocardiogram (ECG) gating in a sagittal oblique volume encompassing the thoracic aorta. The three-directional encoding velocity was equal to 250 cm/s and the scan parameters were as follows: flip angle = 15°, reconstructed voxel size = 1.5 × 2.4 × 1 mm3, echo time = 1.7 ms, repetition time = 4.3 to 4.4 ms, acquisition matrix = 256 × 96 × 136 and views per segment = 2, resulting in acquired temporal resolution = 34.4 to 35.2 ms, which was then reconstructed into 50 frames per cardiac cycle while applying a view sharing technique. Acquisition time was around 10 min. Under-sampled k-space data were reconstructed using an algorithm combining iterative autocalibrating parallel imaging and compressed sensing with nonlinear ℓ1-norm wavelet regularization (L1 SPIR-iT) [26,27,28] with a factor of acceleration of 2 for the slice direction and 2.4 for the phase direction. 4D flow CMR datasets of 13 randomly selected subjects (44 ± 13 years) were also reconstructed using 20 frames per cardiac cycle to study the effect of temporal resolution on aortic PWV estimation. Injection of 0.20 mmol/kg gadolinium contrast agent (gadobenate dimeglumine, Guerbet, France) was performed just prior to 4D flow CMR acquisition.

Blood pressures were measured using a Sphygmocor Xcel (AtCor Medical, Australia) device simultaneously to CMR acquisitions, and central systolic (SBP) and diastolic (DBP) blood pressures were recorded. Pulse pressure (PP=SBP-DBP) was subsequently calculated. All subjects also underwent an applanation tonometry exam immediately after CMR acquisitions to compute the Cf-PWV, as previously described [29], to provide an CMR-independent arterial PWV measure.

In addition, cine balanced steady-state free precession (bSSFP) images were acquired during breath-holds perpendicular to the aorta at the level of pulmonary artery bifurcation to simultaneously image both AAo and DAo as well as in short axis views to cover the whole LV, using the following scan parameters: acquisition matrix = 260 × 192, repetition time = 3.7 ms, echo time = 1.5 ms, flip angle = 50°, pixel size = 0.74 mm × 0.74 mm, slice thickness = 8 mm, views per segment = 12. LV data were analysed using Qmass 6 Software (Medis, Leiden, the Netherlands) while semi-automatically tracing endocardial and epicardial contours on all contiguous short axis slices, resulting in LV end-diastolic (EDV) and end-systolic (ESV) volumes as well as LV mass (LVM). The LV mass-to-volume ratio (LVM/EDV) was calculated and used as a measure of LV remodeling [30].

Aortic bSSFP images were analysed using the validated ArtFun software (Sorbonne Université/U.1146 Inserm) [31], which automatically delineated AAo lumen contours for all phases of the cardiac cycle. Then, local AAo BH-PWV was calculated from AAo distensibility (AAo systolic area-AAo diastolic area)/(AAo diastolic area · central PP) according to the Bramwell-Hill (BH) model [15,16,17] i.e. BH-PWV = 1/ \( \sqrt{\uprho \times \mathrm{distensibility}} \) where ρ is blood density set to 1060 kg.m− 3.

Estimation of aortic pulse wave velocity from 4D flow CMR data

Background phase offsets and phase wrapping were corrected as previously recommended [23]. Besides, to improve the quality of aortic segmentation, a 3D PC CMR angiogram was computed [32] while combining the modulus and velocity images averaged over time phases around the systolic peak, defined as the temporal phase with maximal velocity in the AAo. Aortic volume segmentation from the PC-CMR angiogram was then used to mask velocities through the cardiac cycle and to extract the aortic centerline and subsequently aortic length, using a semi-automated custom segmentation software (Mimosa, Sorbonne Université/Inserm 1146) [33]. Finally, 25 planes which were equally spaced and automatically positioned perpendicularly to the aortic centerline between the distal DAo and AAo, were used to calculate 25 mean velocity curves along the aorta which were interpolated with a time step of 1 ms using a spline function (Fig. 1a).

Fig. 1
figure 1

Aortic pulsed wave velocity (PWV) estimation from 4D flow CMR. a Shared pre-processing step for all aortic PWV (aoPWV) strategies: aortic 3D segmentation, positioning of cross-sectional planes perpendicular to the aortic centreline between the distal descending aorta (1st plane) and proximal ascending aorta (25th plane), extraction of the corresponding 25 mean velocity curves, as well as Di distances between the 1st and the ith planes along the centreline. Of note, mean velocity curves were interpolated with a 1-ms time step using a spline function. b For strategy 1 (S1), aoPWV was defined as the ratio of the distance (D25) between the most distal (#1) and most proximal (#25) planes to the transit time (TT25) estimated from the corresponding normalized velocity waveforms. Transit time estimation methods are described in Fig. 2. c For strategy 2 (S2), intermediate distances (Di) and transit times (TTi) were estimated as in S1 between plane 1 and each successive plane (#i), until the proximal ascending aortic plane is reached. Such process provides the plot at the bottom, which was then linearly fitted (L: TTi = a*Di + b). aoPWV was equal to reciprocal of the slope (a). d For strategy 3 (S3), transit time estimation was not needed since aoPWV was defined from the parameters of the plane (P: a × distance+b × time + c × velocity + d = 0) used to fit the systolic upslope of the normalized velocity curves corresponding to the 25 aortic planes illustrated in panel a

Three different strategies were then used for aoPWV estimation. While the first strategy (Strategy 1) was equivalent to a 2D PC-CMR strategy based on two anatomical locations, strategy 2 and 3 included all the 3D information.

Strategy 1

aoPWV was computed as the distance (D) along the centerline between two extreme planes located in AAo and distal DAo, divided by TT calculated from normalized mean velocity curves at these two locations: aoPWV = D/TT (Fig. 1b). Of note, methods used for TT estimation are described below. Contrary to a through-plane 2D PC-CMR PWV measurement, a more extended segment of the aorta was included to keep PWV Cramer-Rao lower error bound (σc) reasonable, and thus to minimize the error on PWV estimation [12] while considering image characteristics such as temporal and spatial resolutions as well as signal to noise ratio.

$$ {\upsigma}_c\ge \frac{c2}{SNR\times {\varphi}_{max}}\sqrt{\frac{6\times \Delta t\times \Delta x\times {t}_{rise}}{L^3}} $$

σc:Cramer-Rao lower error bound; c: estimation of PWV; ∆t, ∆x : temporal, spatial resolution, respectively; trise: systolic upslope duration; SNR: signal to noise ratio; φmax: phase angle at the maximal velocity; L: vessel length.

Strategy 2

aoPWV estimation used the full 3D anatomical coverage [24, 34]. First, intermediate distances (Di) along the aortic centreline were obtained while considering the distal DAo plane and each successive plane until proximal AAo was reached (Fig. 1a). Second, the corresponding paired normalized mean velocity waveforms were used for TTi estimation. Such intermediate distances and transit times were plotted as TTi according to Di and, aoPWV was equal to the slope of the linear regression plot (Fig. 1c). Of note, methods used for TTi estimation are described below.

Strategy 3

aoPWV was estimated using the full 3D anatomical coverage of the aorta as well, but required no TT calculation [20]. First, normalized mean velocity curves in aortic planes positioned between the proximal AAo and distal DAo were plotted according to time and distance along the aortic centreline (Fig. 1a and d). Then, aoPWV was calculated from the parameters of the plane used to fit the upslope of these curves, which was restricted between the 2nd and 8th decile of the systolic upslope.

Transit time estimation

For Strategy 1 and Strategy 2, three methods of TT estimation previously described in the literature [15, 25] were tested (Fig. 2). The first method (TTc) is based on cross-correlation which was shown to be reproducible, robust and reliable using 2D PC-CMR [35, 36]. The two other methods are based on wavelet (TTw) and Fourier (TTf) transforms, respectively, and were shown to be robust to poor temporal resolution [37, 38]. For all methods, TT is the time delay between two mean velocity curves or their systolic upslope measured at two different locations. Systolic upslope was restricted between its 2nd and the 8th decile. Technical details regarding TT estimates, which are described below, are provided in Fig. 2.

Fig. 2
figure 2

Transit time estimation from 4D flow CMR normalized time-resolved velocity curves. a Transit time (TT) is defined as the time shift needed to maximise the overlap between two normalized aortic velocity curves x(t) and y(t) extracted from two distinct planes positioned along the aortic centreline as illustrated in Fig. 1. Three TT estimation methods are used. b Cross-correlation transit-time (TTc): a time shift (ti), comprised between 0 and the systolic duration is applied to x(t) while maximizing its overlap with y(t) in terms of systolic upslope. Such overlap was iteratively defined by the cross-correlation Cor(ti) between the shifted curve x(t + ti) and y(t) systolic upslopes. TTc was set to the time shift maximizing the cross-correlation function Cor(ti). c Wavelet transform transit-time (TTw): 4th order Gaussian wavelet transform was applied on the normalized mean velocity curves x(t) and y(t) resulting in the provided modulus and phase of the cross-spectrum. TTw is then defined as the sum of such phase weighted by its modulus (see equation), while considering only the systolic upslope. d Fourier transform transit-time (TTf): time shift is modelled by the group delay introduced by a filter, which considers x(t) as an input and y(t) as an output. Normalized mean velocity curves x(t) and y(t) are Fourier transformed into (X(f) and Y(f)) defining the filter transfer function H(f) = Y(f)/X(f). TTf is calculated as the sum of the filter group delay (GD) weighted by the harmonics of the input signal (see equation)

Cross-correlation transit time (TTc)

TT was equal, in the time domain, to the time shift which maximizes the correlation between the two mean velocity curves [39]. The measurement of TTc (Fig. 2b) was performed on normalized mean velocity curves while considering only the systolic upslope [40, 41] to minimize the effects of backward flow, which occurs mostly in late systolic or diastolic phases.

Wavelet transit time (TTw)

TT estimation in the time-frequency domain was derived from the 4-order complex Gaussian wavelet transform, which was initially applied to the normalized mean velocity curves at two aortic locations [37]. The complex cross-spectrum phase difference and modulus derived from such wavelet transforms were then restricted to the systolic upslope, similarly to the cross-correlation method. Finally, TTw was equal to the sum of the cross-spectrum phase difference weighted by its modulus (Fig. 2c) [37].

Fourier transit time (TTf)

TT estimation was modelled by the group delay of a filter which takes as input and output Fourier transforms (X(f) and Y(f)) of two normalized mean velocity curves (x(t) and y(t), respectively) measured at two aortic locations. Such filter transfer function was calculated as H(f) = Y(f)/X(f). TTf was then equal to the group delay weighted by the harmonics of the input signal (Fig. 2d). In this frequency domain-based method [38], the entire time interval rather than systolic upslope was considered.

Inter-observer reproducibility analysis

The only source of variability was the aortic segmentation, while the remaining steps for 4D flow CMR aoPWV estimation were fully automated. Accordingly, such segmentation along with computation of aoPWV using the above-mentioned strategies were performed on a subgroup of 15 randomly selected subjects (47 ± 22 years), in a random order by two independent operators (SHGS and KB with 2-year experience in 4D flow image processing), both blinded to subjects characteristics and to each other’s segmentation and quantitative results.

Statistical analysis

Basic characteristics and PWV estimates are provided as median and interquartile range. The study population was divided into two subgroups according to age (< or ≥ 50 years) and Wilcoxon rank-sum test was used for comparison between the two age subgroups. Univariate linear regression was performed to study the relationships of 4D flow CMR aoPWV estimates, with Cf-PWV, BH-PWV, age, and LV mass-to-volume ratio. Regarding associations with LV mass-to-volume ratio, further multivariate models including potential confounders such as age, body mass index (BMI), SBP and gender were studied. Correlation coefficients R and estimated coefficient β are provided. Bland-Altman analyses were performed for: 1) comparison between 4D flow CMR methods and Cf-PWV as well as BH-PWV, to provide the positioning of 4D flow CMR measures against well-established PWV measures, 2) studying 4D flow CMR aoPWV inter-operator reproducibility and the effect of lower temporal resolution by comparing results of the 20 and 50 phases reconstructions, 3) comparison of the 4D flow CMR aoPWV method that was shown to be superior to the other methods based on the analyses described above, against the remaining 4D flow CMR aoPWV methods. For all Bland-Altman analyses, mean biases (MB), limits of agreements (LA) defined as mean bias±1.96 x Standard deviation [42] were provided. Intraclass correlation coefficients (ICC) were also calculated. Finally, the Wilcoxon rank-sum test was used to test for statistical significance of the differences between aoPWV values obtained by the two observers or for the two reconstructions. A result was reported as significant if p < 0.05. Statistical analyses were performed in Matlab (Mathworks, Natick, Massachusetts, USA).

Results

Table 1 summarizes subjects characteristics. As expected, SBP and PP were significantly higher for older subjects when compared to younger subjects. The effect of age was also observed on Cf-PWV, BH-PWV and LV mass-to-volume ratio (LVM/EDV), which were all significantly higher in the older subjects. All 4D flow CMR aoPWV values are also presented in Table 1. They all showed a significant increase (p < 0.001) between the younger and older groups.

Table 1 Subjects characteristics and pulse wave velocity estimates

Association between 4D flow CMR aoPWV and Cf-PWV as well as BH-PWV

Correlation coefficients for comparisons of all 4D flow-derived aoPWV estimates against reference tonometric Cf-PWV and ascending aorta BH-PWV are summarized in Table 2 and the corresponding linear regression plots are provided in Fig. 3. All 4D flow aoPWV estimates were significantly correlated (p < 0.05) with Cf-PWV as well as AAo BH-PWV, with the highest correlation coefficients obtained when using Strategy 2 along with the wavelet-based TT estimation (S2-TTw PWV: R = 0.62, p < 0.001 with Cf-PWV; R = 0.65, p < 0.001 with BH-PWV) and Strategy 3 (S3 PWV: R = 0.61, p < 0.001 with Cf-PWV; R = 0.64, p < 0.001 with BH-PWV).

Table 2 Associations of 4D flow CMR aoPWV methods with Cf-PWV, BH-PWV, age and LV mass-to-volume ratio
Fig. 3
figure 3

Associations between 4D flow CMR aortic PWV and carotid-femoral (Cf) PWV. Linear regressions (solid lines) and confidence intervals (shaded blue area) are displayed. Linear regression equations are provided. S1, S2 and S3: Strategies 1, 2 and 3. TTc, TTw and TTf: transit times estimated using cross-correlation, wavelets and Fourier transforms, respectively. Correlation coefficients (R) are provided along with their levels of significance, which were indicated by *** for p < 0.001, ** for p < 0.01, * for p < 0.05

Associations of 4D flow CMR aoPWV with age and left ventricular mass-to-volume ratio

Correlation coefficients for associations of all 4D flow CMR aoPWV values with age and with LV mass-to-volume ratio are shown in Table 2. All 4D flow-derived aoPWV measures were significantly associated with age, with the highest correlations obtained for S2-TTw (R = 0.77, p < 0.001) and S3 (R = 0.76, p < 0.001). Of note, such correlations were in the same range as the correlations between age and reference Cf-PWV (R = 0.72, p < 0.001) as well as BH-PWV (R = 0.79, p < 0.001). Interestingly, association with age remained significant even in the elderly group for both S2-TTw (R = 0.59, p < 0.001) and S3 (R = 0.59, p < 0.001) aoPWV measures. Significant correlations with age in univariate analysis remained significant (p < 0.001) after adjustment for BMI, gender and SBP without other significant correlates.

For associations with LV mass-to-volume ratio, 4D flow CMR aoPWV obtained using S2-TTw (R = 0.52, p < 0.001) and S3 (R = 0.47, p < 0.001) resulted in the highest correlations in univariate analysis. Such correlations were even slightly higher than those obtained for the reference Cf-PWV (R = 0.39, p < 0.01) and equivalent to those obtained for BH-PWV (R = 0.52, p < 0.001). The associations with LV mass-to-volume ratio obtained for S2-TTw, S3, and Cf-PWV remained significant after adjustment for age, gender, BMI, SBP (Table 3). Of note, gender was also a significant correlate.

Table 3 Independent correlates of left ventricular mass-to-volume ratio

Inter-observer reproducibility and effect of temporal resolution

Results of inter-observer reproducibility are summarized in the left part of Table 4. For all methods, there were no significant differences between 4D flow CMR aoPWV values measured by the two operators (p ≥ 0.38). Bland-Altman mean biases and limits of agreement indicated a higher reproducibility of the 3D-based strategies (S2 and S3), as compared with the 2D-based strategy (S1).

Table 4 Inter-observer reproducibility and effect of 20- vs. 50-time reconstructed frames temporal resolution

Analysis of the effect of reconstructed temporal resolution is summarized in the right part of Table 4, revealing that approaches using computation only in time domain (S1-TTc, S2-TTc, and S3) were less robust to the 20-phase reconstruction (ICC < 0.82) than frequency (S1-TTf, S2-TTf) or time-frequency (S1-TTw, S2-TTw) domain (ICC > 0.91) methods. Of note, at each given transit time method, the strategy with a 3D coverage performed better than the 2D plans strategy.

Comparisons between the various PWV estimates

Table 5 summarizes results of the Bland-Altman analyses for comparisons of the 4D flow CMR aoPWV methods with Cf-PWV and BH-PWV. The corresponding plots are shown and commented in the Additional file 1. For comparisons against Cf-PWV, mean bias was close to zero or negative, in line with higher stiffness of peripheral arteries as compared with central elastic arteries. For comparisons against ascending aortic BH-PWV, mean bias was close to zero or positive, highlighting the highest elasticity of the most proximal aortic segment. Comparison between the two 4D flow CMR aoPWV methods (S2-TTw and S3) that were shown to be superior to other estimates, in terms of reproducibility as well as associations with Cf-PWV, BH-PWV, age, and LV mass-to-volume ratio, resulted in the highest ICC and the narrowest limits of agreement (Fig. 4).

Table 5 Bland-Altman analyses for comparisons of the 4D flow CMR aoPWV methods with Cf-PWV and BH-PWV
Fig. 4
figure 4

Bland-Altman analysis between S2-TTw and the remaining 4D flow CMR-derived aortic PWV. Mean bias (solid lines), limits of agreements (shaded light blue region and dotted lines) and confidence intervals (shaded green region) are showed. Intra-class correlation coefficients (ICC) are provided above each plot. S1, S2 and S3: Strategies 1, 2 and 3, respectively. TTc, TTw and TTf: transit times estimated using cross-correlation, wavelets and Fourier transforms, respectively

Discussion

This study provides a comparison of the main methods available in the literature for aortic PWV estimation, using 4D flow CMR in 47 healthy subjects. The significant associations with age and the non-invasive reference applanation tonometry-derived Cf-PWV (p < 0.001) demonstrated the consistency of all 4D flow CMR-derived aoPWV estimates. Methods taking into account the whole 3D aortic spatial coverage, specifically the method based on the wavelet transform for transit time estimation were found to be superior to the other approaches, as revealed by 1) their stronger associations with Cf-PWV, Bramwell-Hill model-derived AAo PWV, age, and LV-mass-to-volume ratio as well as 2) their higher reproducibility and robustness to lower reconstructed temporal resolution. Interestingly, correlations of 4D flow CMR aoPWV methods, which account for the full 3D coverage, with age and LV mass-to-volume ratio were in the same range or even slightly higher than those obtained when using Cf-PWV, and remained significant after adjustment for the main confounders.

Aortic PWV values obtained in our study from 4D flow data are in a similar range using all methods. Furthermore, comparison against the widely available 2D PC-CMR aortic arch PWV values estimated in large populations in the literature revealed no substantial differences. Indeed, normal values for aortic arch PWV summarized by Kawel-Boehm et al. [43] were 3.9 ± 1.1 m/s for age range 30–39 years, 5.6 ± 1.4 m/s for 40–49 years, 7.2 ± 2.3 m/s for 50–59 years, 9.7 ± 2.9 m/s for 60–69 years, 11.1 ± 4.6 m/s for age ≥ 70 years. Normal values for 4D flow CMR-derived aoPWV are not available yet since the majority of previous studies was performed on small groups or pathological individuals while using various methods. Nevertheless, despite differences in population age, our values were in the same range or only slightly higher than those provided in a recent 4D flow study [44] while using similar TT-based approaches (n = 8, age = 23 ± 2 years: PWV = 5.7 ± 0.7 m/s when using Fourier analysis and PWV = 5.5 ± 0.7 m/s when using cross-correlation; n = 8, age = 58 ± 2 years: PWV = 9.3 ± 1.3 m/s when using Fourier analysis and PWV = 8.9 ± 1.4 m/s when using cross-correlation).

Our choice of the tested methods for aoPWV estimation in this study was based on previous 2D PC-CMR and 4D flow CMR studies. Indeed they have demonstrated that: 1) approaches based on a single point of the flow curve (foot or peak) are hampered by low velocity to noise ratio or by the effects of wave reflection [24, 36, 45], 2) frequency and time-frequency domain methods for TT estimation are more robust to low temporal resolution [37, 38], and 3) approaches taking into account volumetric flow data along the aorta are more robust than methods considering only two measurement sites [20, 24]. The high reproducibility of PWV previously shown in 2D PC-CMR studies was not found in our 2D-like strategy (S1), which resulted in sizeable Bland-Altman limits of agreements in our data. Such lower performances of the 2D-like strategies in our study might be explained by the static nature of the 4D flow CMR aortic lumen segmentation through time, contrary to an automated dynamic time-resolved segmentation in 2D PC-CMR, which would hamper the AAo velocity curves, as well as by the lower spatial resolution of 4D flow as compared to 2D PC-CMR data, which would highly affect the distal descending aorta because of its small cross-section.

In agreement with previous findings [36, 37], systolic upslope-based methods were more consistent than wave-based methods especially when using the 3D strategy, as revealed by the comparison between wavelet or cross-correlation and Fourier approaches in terms of associations with age and LV mass to volume ratio. This can be explained by the fact that the early systolic upslope is less distorted by wave reflexion than late systolic and diastolic phases of the flow curve [35, 36]. Besides, methods considering the volumetric coverage of 4D flow CMR data were more reliable than those based on two planes only.

Original features of our study include semi-automated segmentation of the aorta and an automated positioning of flow measurement planes perpendicular to the centreline. In addition, comparisons against the CMR-independent tonometry Cf-PWV and 4D flow CMR-independent ascending aortic BH-PWV measures, as well as the assessment of physiological associations with LV mass-to-volume ratio and age were reported. Our aoPWV measurements confirmed physiological knowledge on arterial stiffening gradient from the central aorta to peripheral arteries, resulting in the lowest values for the ascending aortic Bramwell-Hill method, intermediate values for the 4D flow CMR methods over the whole aorta, and the highest values for the carotid to femoral arteries PWV. This stiffness gradient phenomenon was more marked in younger subjects with highly elastic central arteries than in elderly subjects, because of stiffness homogenisation from central arteries towards the periphery with ageing.

Furthermore, comparison to Cf-PWV and BH-PWV revealed the superiority of the plane fitting method as well as wavelet TT-based approach previously shown using 2D-PC CMR to be more robust to low temporal resolution [37]. Our findings further confirm such robustness of the wavelet-based method to 20- vs. 50-reconstructed time frames. In addition, such method was strongly associated with age even in subjects ≥50 years, in whom we expect increased aortic stiffness and thus decreased TT when compared to younger subjects. Although the wavelet-based approach resulted in the highest performances when compared to the plane fitting approach, its implementation requires tuning of parameters such as the mother wavelet and sampling frequency. However, one might highlight that the same parameters than those previously described using 2D-PC CMR data [37] were used in each subject for the wavelet method in the present study.

Reliability of our 4D flow CMR aoPWV measurement was also demonstrated by the positive association with LV mass-to-volume ratio. Indeed, it is known that in the process of healthy aging, arterial stiffening is associated with increased LV afterload and subsequent LV hypertrophic remodeling, which was measured in our study through the LV mass to end-diastolic volume ratio [30]. Such association was independent of age, systolic blood pressure, BMI, and gender, revealing that 4D flow CMR aoPWV is an independent marker of LV alteration in the process of healthy aging. Gender was also an independent correlate of LV mass-to-volume ratio, in agreement with a previous study [46]. Accordingly, evaluation of aortic stiffness using 4D flow CMR can provide mechanistic knowledge to improve understanding of ventricular-arterial coupling.

The main limitation of our study is the lack of a direct aortic PWV gold standard measurement as provided by catheterization [10]. However such invasive procedure was not feasible in healthy subjects. Moreover, associations with the non-invasive reference Cf-PWV [1, 7, 29], and with physiological criteria such as age and LV mass-to-volume ratio were used to compare the performances of 4D flow CMR aoPWV methods. Since 4D flow CMR is limited by low spatial resolution and signal to noise ratio for low velocities, time-resolved segmentation was not available. Accordingly, a fixed segmentation on an enhanced aortic angiogram, reconstructed from peak systolic phases, was used along with mean velocity curves rather than flow curves. In the future, a time-resolved segmentation would be necessary for an accurate estimation of aortic flow along the cardiac cycle. Such segmentation would require an evolvement of 4D flow CMR sequences with the use of higher spatial resolution and multiple encoding velocities to improve velocity to noise ratio especially in regions with low velocities or the use of motion-compensated compressed-sensing techniques which were shown to improve the quality of 4D flow images [47, 48]. To enhance contrast and signal to noise ratio, contrast agent is often used in 4D flow CMR. However, since standard CMR exams in clinical routine usually include LGE and/or post-contrast T1, 4D flow CMR acquisition can be interleaved while waiting for such tissue characterization sequences to be acquired. Another limitation is the lack of test-retest variability assessment of the 4D flow CMR aoPWV due to non-available data. However, one might highlight that the main goal of our study was to isolate the technical reliability of 4D flow CMR aoPWV from the robustness of 4D flow CMR to hemodynamic changes over time. Thus, the proposed 4D flow CMR techniques were rather compared in terms of inter-observer reproducibility and associations with 4D flow CMR-independent aoPWV measures such as the tonometric Cf-PWV and the cine bSSFP-derived BH-PWV as well as physiological association with age and LV mass-to-volume ratio.

Conclusion

Aortic PWV estimation in healthy subjects from 4D flow CMR data while considering the entire volumetric coverage of the aorta and a time-frequency wavelet approach for transit time calculation was reproducible, robust to lower reconstructed temporal resolution and reliable, as revealed by the strong associations with the well-established reference Cf-PWV, BH-PWV and physiological criteria such as age and LV mass-to-volume ratio. The LV and the aorta being simultaneously acquired in the same hemodynamic conditions when using 4D flow CMR, the derived aortic PWV might help for a better understanding of LV – aortic coupling.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

AAo:

Ascending aorta

aoPWV:

Aortic pulse wave velocity

BH:

Bramwell-Hill

BMI:

Body mass index

bSSFP:

Balanced steady-state free precession

Cf:

Carotid-femoral

CMR:

Cardiovascular magnetic resonance imaging

D:

Distance

DAo:

Descending aorta

DBP:

Diastolic blood pressure

ECG:

Electrocardiogram

EDV:

End-diastolic volume

ESV:

End-systolic volume

ICC:

Intra-class correlation coefficient

LV:

Left ventricle/left ventricular

LVM:

Left ventricular mass

PC:

Phase-contrast

PP:

Pulse pressure

PWV:

Pulse wave velocity

SBP:

Systolic blood pressure

TT:

Transit time

References

  1. Laurent S, Boutouyrie P, Asmar R, Gautier I, Laloux B, Guize L, et al. Aortic stiffness is an independent predictor of all-cause and cardiovascular mortality in hypertensive patients. Hypertension. 2001;37:1236–41.

    Article  CAS  PubMed  Google Scholar 

  2. Safar ME, London GM. Therapeutic studies and arterial stiffness in hypertension: recommendations of the European Society of Hypertension. The clinical Committee of Arterial Structure and Function. Working group on vascular structure and function of the European Society of Hypertension. J Hypertens. 2000;18:1527–35.

    Article  CAS  PubMed  Google Scholar 

  3. Meaume S, Benetos A, Henry OF, Rudnichi A, Safar ME. Aortic pulse wave velocity predicts cardiovascular mortality in subjects >70 years of age. Arterioscler Thromb Vasc Biol. 2001;21:2046–50.

    Article  CAS  PubMed  Google Scholar 

  4. Ben-Shlomo Y, Spears M, Boustred C, May M, Anderson SG, Benjamin EJ, et al. Aortic pulse wave velocity improves cardiovascular event prediction: an individual participant meta-analysis of prospective observational data from 17,635 subjects. J Am Coll Cardiol. 2014;63:636–46.

    Article  PubMed  Google Scholar 

  5. Sutton-Tyrrell K, Najjar SS, Boudreau RM, Venkitachalam L, Kupelian V, Simonsick EM, et al. Elevated aortic pulse wave velocity, a marker of arterial stiffness, predicts cardiovascular events in well-functioning older adults. Circulation. 2005;111:3384–90.

    Article  PubMed  Google Scholar 

  6. Willum-Hansen T, Staessen JA, Torp-Pedersen C, Rasmussen S, Thijs L, Ibsen H, et al. Prognostic value of aortic pulse wave velocity as index of arterial stiffness in the general population. Circulation. 2006;113:664–70.

    Article  PubMed  Google Scholar 

  7. Collaboration TRV for AS. Determinants of pulse wave velocity in healthy people and in the presence of cardiovascular risk factors: establishing normal and reference values. Eur Heart J. 2010;31:2338–50.

    Article  Google Scholar 

  8. Alecu C, Labat C, Kearney-Schwartz A, Fay R, Salvi P, Joly L, et al. Reference values of aortic pulse wave velocity in the elderly. J Hypertens. 2008;26:2207–12.

    Article  CAS  PubMed  Google Scholar 

  9. Van Bortel LM, Balkestein EJ, van der Heijden-Spek JJ, Vanmolkot FH, Staessen JA, Kragten JA, et al. Non-invasive assessment of local arterial pulse pressure: comparison of applanation tonometry and echo-tracking. J Hypertens. 2001;19:1037–44.

    Article  PubMed  Google Scholar 

  10. Latham RD, Westerhof N, Sipkema P, Rubal BJ, Reuderink P, Murgo JP. Regional wave travel and reflections along the human aorta: a study with six simultaneous micromanometric pressures. Circulation. 1985;72:1257–69.

    Article  CAS  PubMed  Google Scholar 

  11. Grotenhuis HB, Westenberg JJM, Steendijk P, van der Geest RJ, Ottenkamp J, Bax JJ, et al. Validation and reproducibility of aortic pulse wave velocity as assessed with velocity-encoded MRI. J Magn Reson Imaging. 2009;30:521–6.

    Article  PubMed  Google Scholar 

  12. Bolster BD, Atalar E, Hardy CJ, McVeigh ER. Accuracy of arterial pulse-wave velocity measurement using MR. J Magn Reson Imaging. 1998;8:878–88.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Hickson SS, Butlin M, Graves M, Taviani V, Avolio AP, McEniery CM, et al. The relationship of age with regional aortic stiffness and diameter. JACC Cardiovasc Imaging. 2010;3:1247–55.

    Article  PubMed  Google Scholar 

  14. Vulliémoz S, Stergiopulos N, Meuli R. Estimation of local aortic elastic properties with MRI. Magn Reson Med. 2002;47:649–54.

    Article  PubMed  Google Scholar 

  15. Dogui A, Kachenoura N, Frouin F, Lefort M, De Cesare A, Mousseaux E, et al. Consistency of aortic distensibility and pulse wave velocity estimates with respect to the Bramwell-Hill theoretical model: a cardiovascular magnetic resonance study. J Cardiovasc Magn Reson. 2011;13:11.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Westenberg JJ, van Poelgeest EP, Steendijk P, Grotenhuis HB, Jukema J, de Roos A. Bramwell-Hill modeling for local aortic pulse wave velocity estimation: a validation study with velocity-encoded cardiovascular magnetic resonance and invasive pressure assessment. J Cardiovasc Magn Reson. 2012;14:2.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Bramwell JC, Hill AV. The velocity of the pulse wave in man. Proc R Soc Lond B Biol Sci. 1922;93:298–306.

    Article  Google Scholar 

  18. Stankovic Z, Allen BD, Garcia J, Jarvis KB, Markl M. 4D flow imaging with MRI. Cardiovasc Diagn Ther. 2014;4:173–92.

    PubMed  PubMed Central  Google Scholar 

  19. Markl M, Schnell S, Barker AJ. 4D flow imaging: current status to future clinical applications. Curr Cardiol Rep. 2014;16:481.

    Article  PubMed  Google Scholar 

  20. Markl M, Wallis W, Strecker C, Gladstone BP, Vach W, Harloff A. Analysis of pulse wave velocity in the thoracic aorta by flow-sensitive four-dimensional MRI: reproducibility and correlation with characteristics in patients with aortic atherosclerosis. J Magn Reson Imaging. 2012;35:1162–8.

    Article  PubMed  Google Scholar 

  21. Guala A, Camporeale C, Ridolfi L, Mesin L. Non-invasive aortic systolic pressure and pulse wave velocity estimation in a primary care setting: an in silico study. Med Eng Phys. 2017;42:91–8.

    Article  PubMed  Google Scholar 

  22. Harloff A, Mirzaee H, Lodemann T, Hagenlocher P, Wehrum T, Stuplich J, et al. Determination of aortic stiffness using 4D flow cardiovascular magnetic resonance - a population-based study. J Cardiovasc Magn Reson. 2018;20:43.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Dyverfeldt P, Bissell M, Barker AJ, Bolger AF, Carlhäll C-J, Ebbers T, et al. 4D flow cardiovascular magnetic resonance consensus statement. J Cardiovasc Magn Reson. 2015;17:72.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Markl M, Wallis W, Brendecke S, Simon J, Frydrychowicz A, Harloff A. Estimation of global aortic pulse wave velocity by flow-sensitive 4D MRI. Magn Reson Med. 2010;63:1575–82.

    Article  PubMed  Google Scholar 

  25. Wentland AL, Wieben O, François CJ, Boncyk C, Rio AMD, Johnson KM, et al. Aortic pulse wave velocity measurements with Undersampled 4D flow-sensitive MRI: comparison to 2D and algorithm determination. J Magn Reson Imaging. 2013;37:853–9.

    Article  PubMed  Google Scholar 

  26. Vasanawala SS, Alley MT, Hargreaves BA, Barth RA, Pauly JM, Lustig M. Improved pediatric MR imaging with compressed sensing. Radiology. 2010;256:607–16.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Lustig M, Pauly JM. SPIRiT: iterative self-consistent parallel imaging reconstruction from arbitrary k-space. Magn Reson Med. 2010;64:457–71.

    PubMed  PubMed Central  Google Scholar 

  28. Vasanawala S, Murphy M, Alley M, Lai P, Keutzer K, Pauly J, et al. Practical parallel imaging compressed sensing MRI: summary of two years of experience in accelerating body MRI of pediatric patients. Proc IEEE Int Symp Biomed Imaging. 2011;2011:1039–43. https://doi.org/10.1109/ISBI.2011.5872579.

  29. Laurent S, Cockcroft J, Van Bortel L, Boutouyrie P, Giannattasio C, Hayoz D, et al. Expert consensus document on arterial stiffness: methodological issues and clinical applications. Eur Heart J. 2006;27:2588–605.

    Article  PubMed  Google Scholar 

  30. Cheng S, Fernandes VRS, Bluemke DA, McClelland RL, Kronmal RA, Lima JAC. Age-related left ventricular remodeling and associated risk for cardiovascular outcomes: the multi-ethnic study of atherosclerosis. Circ Cardiovasc Imaging. 2009;2:191–8.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Herment A, Kachenoura N, Lefort M, Bensalah M, Dogui A, Frouin F, et al. Automated segmentation of the aorta from phase contrast MR images: validation against expert tracing in healthy volunteers and in patients with a dilated aorta. J Magn Reson Imaging. 2010;31:881–8.

    Article  PubMed  Google Scholar 

  32. Markl M, Harloff A, Bley TA, Zaitsev M, Jung B, Weigang E, et al. Time-resolved 3D MR velocity mapping at 3T: improved navigator-gated assessment of vascular anatomy and blood flow. J Magn Reson Imaging. 2007;25:824–31.

    Article  PubMed  Google Scholar 

  33. Dietenbeck T, Craiem D, Rosenbaum D, Giron A, De Cesare A, Bouaou K, et al. 3D aortic morphology and stiffness in MRI using semi-automated cylindrical active surface provides optimized description of the vascular effects of aging and hypertension. Comput Biol Med. 2018;103:101–8.

    Article  PubMed  Google Scholar 

  34. Spottiswoode B, Stalder AF, Gulsun MA, Campione KF, Carr M, Wasielewski M, et al. Fast semi-automated analysis of pulse wave velocity in the thoracic aorta using high temporal resolution 4D flow MRI. J Cardiovasc Magn Reson. 2013;15:P87.

    Article  PubMed Central  Google Scholar 

  35. Ibrahim E-SH, Johnson KR, Miller AB, Shaffer JM, White RD. Measuring aortic pulse wave velocity using high-field cardiovascular magnetic resonance: comparison of techniques. J Cardiovasc Magn Reson. 2010;12:26.

    Article  PubMed Central  Google Scholar 

  36. Dogui A, Redheuil A, Lefort M, DeCesare A, Kachenoura N, Herment A, et al. Measurement of aortic arch pulse wave velocity in cardiovascular MR: comparison of transit time estimators and description of a new approach. J Magn Reson Imaging. 2011;33:1321–9.

    Article  PubMed  Google Scholar 

  37. Bargiotas I, Mousseaux E, Yu W-C, Venkatesh BA, Bollache E, de Cesare A, et al. Estimation of aortic pulse wave transit time in cardiovascular magnetic resonance using complex wavelet cross-spectrum analysis. J Cardiovasc Magn Reson. 2015;17:65.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Meloni A, Zymeski H, Pepe A, Lombardi M, Wood JC. Robust estimation of pulse wave transit time using group delay. J Magn Reson Imaging. 2014;39:550–8.

    Article  PubMed  Google Scholar 

  39. Ou P, Celermajer DS, Raisky O, Jolivet O, Buyens F, Herment A, et al. Angular (gothic) aortic arch leads to enhanced systolic wave reflection, central aortic stiffness, and increased left ventricular mass late after aortic coarctation repair: evaluation with magnetic resonance flow mapping. J Thorac Cardiovasc Surg. 2008;135:62–8.

    Article  PubMed  Google Scholar 

  40. Westerhof BE, Guelen I, Westerhof N, Karemaker JM, Avolio A. Quantification of wave reflection in the human aorta from pressure alone. Hypertension. 2006;48:595–601.

    Article  CAS  PubMed  Google Scholar 

  41. Bensalah MZ, Bollache E, Kachenoura N, Giron A, Cesare AD, Macron L, et al. Geometry is a major determinant of flow reversal in proximal aorta. Am J Physiol - Heart Circ Physiol. 2014;306:H1408–16.

    Article  CAS  PubMed  Google Scholar 

  42. Bland JM, Altman D. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;327:307–10.

    Article  Google Scholar 

  43. Kawel-Boehm N, Maceira A, Valsangiacomo-Buechel ER, Vogel-Claussen J, Turkbey EB, Williams R, et al. Normal values for cardiovascular magnetic resonance in adults and children. J Cardiovasc Magn Reson. 2015;17:29.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Dyverfeldt P, Ebbers T, Länne T. Pulse wave velocity with 4D flow MRI: systematic differences and age-related regional vascular stiffness. Magn Reson Imaging. 2014;32:1266–71.

    Article  PubMed  Google Scholar 

  45. Fielden SW, Fornwalt BK, Jerosch-Herold M, Eisner RL, Stillman AE, Oshinski JN. A new method for the determination of aortic pulse wave velocity using cross-correlation on 2D PCMR velocity data. J Magn Reson Imaging. 2008;27:1382–7.

    Article  PubMed  Google Scholar 

  46. Douglas PS, Katz SE, Weinberg EO, Chen MH, Bishop SP, Lorell BH. Hypertrophic remodeling: gender differences in the early response to left ventricular pressure overload. J Am Coll Cardiol. 1998;32:1118–25.

    Article  CAS  PubMed  Google Scholar 

  47. Cheng JY, Hanneman K, Zhang T, Alley MT, Lai P, Tamir JI, et al. Comprehensive motion-compensated highly accelerated 4D flow MRI with ferumoxytol enhancement for pediatric congenital heart disease. J Magn Reson Imaging. 2016;43:1355–68.

    Article  PubMed  Google Scholar 

  48. Cheng JY, Zhang T, Ruangwattanapaisarn N, Alley MT, Uecker M, Pauly JM, et al. Free-breathing pediatric MRI with nonrigid motion correction and acceleration. J Magn Reson Imaging. 2015;42:407–20.

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

We would like to thank Dr. Marcus Alley, Senior Research Scientist in the Department of Radiology and Radiological Sciences Laboratory of the Stanford University School of Medicine, for his input regarding the technical details of the 4D flow MRI acqusition sequence

Funding

We acknowledge the FRM project ING20150532487 for funding Kevin Bouaou and ESME-Sudria for funding Sophia Houriez--Gombaud-Saintonge as well as the ECOS-SUD project number A15S04 (France-Argentina) exchange grant for funding a fruitful exchange around 4D flow CMR data processing.

Author information

Authors and Affiliations

Authors

Contributions

All authors contributed significantly to this manuscript. All authors: manuscript drafting or manuscript revision for important intellectual content. All authors: approval of final version of submitted manuscript. All authors: literature research. SH, EM, GS, UG, AG, EMe, EB YC, NK study concepts/study design or data acquisition or data and statistical analysis or findings interpretation. SH, TD, AD, KB, IB, NK, software design for quantification and analysis. YC and NK contributed equally to this work.

Corresponding author

Correspondence to Nadjia Kachenoura.

Ethics declarations

Ethics approval and consent to participate

This study conformed to local ethics committee regulations. Ethics committee approval was waived for this study type. Signed informed consent was given by all participants.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1.

Bland-Altman plots for comparisons of 4D flow MRI aoPWV with Cf-PWV and BH-PWV.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Houriez--Gombaud-Saintonge, S., Mousseaux, E., Bargiotas, I. et al. Comparison of different methods for the estimation of aortic pulse wave velocity from 4D flow cardiovascular magnetic resonance. J Cardiovasc Magn Reson 21, 75 (2019). https://doi.org/10.1186/s12968-019-0584-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12968-019-0584-x

Keywords