A granular approach to improve reproducibility of the echocardiographic assessment of paravalvular regurgitation after TAVI

Paravalvular leak (PVL) after transcatheter aortic valve implantation (TAVI) is challenging to quantitate. Transthoracic echocardiography (TTE) is the main tool used for the assessment of PVL but is modestly reproducible. We sought to develop a reproducible echocardiographic approach to assess PVL in the post-TAVI setting. Four observers independently analyzed eleven parameters of PVL severity in 50 pre-discharge TTE studies performed after TAVI. The parameters included color-Doppler parameters [jet circumferential extent (CE) and planimetered vena contracta area in the short-axis view and jet breadth and qualitative features in the long-axis views], continuous-wave Doppler parameters [jet velocity time integral (VTI) and pressure half time (PHT)], quantitative Doppler parameters (regurgitation volume and fraction and effective regurgitant orifice area), aortic diastolic flow reversal and valve stent eccentricity. Intraclass correlation coefficient (ICC) and coefficient of variation (CV) for numerical parameters and kappa coefficient (κ) for categorical parameters were calculated for inter- and intra-observer comparisons. Inter-observer ICC was highest and CV lowest for CE (0.88 and 0.36), jet origin breadth (0.82 and 0.39), jet qualitative features in long-axis views (0.87 and 0.26), jet VTI (0.87 and 0.04) and PHT (0.73 and 0.10). Similar results were found in intra-observer comparisons. A 2-step granular approach combining the most reproducible parameters was used to grade PVL by the four observers. Inter-observer agreement was achieved in 86 % of cases (κ = 0.79). Combining color Doppler and continuous wave Doppler parameters in a granular algorithm yields excellent reproducibility of PVL assessment by TTE. Electronic supplementary material The online version of this article (doi:10.1007/s10554-016-0947-4) contains supplementary material, which is available to authorized users.


Introduction
Transcatheter aortic valve implantation (TAVI) is the treatment of choice for inoperable, a recommended alternative to surgery in high-risk, and a potential option in intermediate-risk patients with symptomatic severe aortic stenosis [1].
Paravalvular leak (PVL) is an important limitation of TAVI as compared to surgical valve replacement [2]. Proper annular sizing [3,4] and the use of more efficient paravalvular sealing technologies [5][6][7][8] led to a significant reduction in the incidence of greater than mild PVL. However, mild PVL is still a common complication of the second/third generation transcatheter aortic valves [5][6][7][8], and has been linked to worse prognosis [9]. Moreover, as TAVI extends to younger patients, more bicuspid anatomy and native Abstract Paravalvular leak (PVL) after transcatheter aortic valve implantation (TAVI) is challenging to quantitate. Transthoracic echocardiography (TTE) is the main tool used for the assessment of PVL but is modestly reproducible. We sought to develop a reproducible echocardiographic approach to assess PVL in the post-TAVI setting. Four observers independently analyzed eleven parameters of PVL severity in 50 pre-discharge TTE studies performed after TAVI. The parameters included color-Doppler parameters [jet circumferential extent (CE ) and planimetered vena contracta area in the short-axis view and jet breadth and qualitative features in the long-axis views], continuous-wave Doppler parameters [jet velocity time integral (VTI) and pressure half time (PHT)], quantitative Doppler parameters (regurgitation volume and fraction and effective regurgitant orifice area), aortic diastolic flow reversal and valve stent eccentricity. Intraclass correlation coefficient (ICC) and coefficient of

Methods
The study protocol has been approved by the institutional review board and all patients provided a written informed consent. The study consisted of three phases. In the first phase, 50 randomly-selected post-TAVI transthoracic 2D echocardiograms were independently analyzed by four cardiologists (BR, E S, MA and OS) of variable experience (in echocardiography; 5-19 years and in analysis of TAVI echocardiograms; 1-10 years) blinded to patients' clinical and procedural data. A summary (mean ± standard deviation) of the individual-observer measurements is provided in Table S1. In 35 echocardiograms, reread by the same observer (BR) was performed at a median interval of 5 months to investigate intra-observer reproducibility. Eleven parameters of PVL severity (Table 1) were analyzed in accordance with the guidelines of the American Society of Echocardiography (ASE) and the European association of valvular regurgitation will be met increasing the potential risk of PVL [10,11]. Furthermore, an excellent paravalvular sealing (at least comparable to surgical bioprosthesis) will be a prerequisite before lower risk patients can be offered TAVI as a recommended option.
Recently, PVL has been reported to regress up to 1 year compared with discharge after TAVI with the self-expanding CoreValve [12]. On the other hand, structural deterioration and new onset valve regurgitation are being increasingly reported [13], further emphasizing the importance of reproducible long-term surveillance.
We sought to investigate and propose an approach to improve the reproducibility of the echocardiographic assessment of PVL severity. PVL jet arc circumferential extent (in degrees as a fraction of 360°) and planimetered area were measured at the plane of the valve stent inflow edge. Care has been exercised to avoid measuring low velocity (laminar) flow and to include the sum of the separate jets, not the paravalvular arc which includes the non-regurgitant space between jets PVL short-axis area Long-axis views Jet origin breadth PVL jet origin breadth is equivalent to the vena contracta of a transvalvular aortic regurgitation. The sum and the average of measurements from the anterior and posterior sides in the PLAX, apical 5-chamber and 3-chamber views (6 sites in total) were calculated. Ratio of the average jet breadth to the LVOT height was used in the final grading of PVL severity Qualitative jet features From the 6 long-axis sites, jet features were assessed and a score (0-none, 1-trace or 2-significant) was accordingly given to the PVL jet; where a jet score of "2" indicates a sizable jet width with continuous turbulence from jet origin to valve stent inflow edge.

Aortic flow (Pulsed-wave Doppler)
Diastolic flow reversal Descending thoracic and abdominal aortic diastolic flow reversal was sought for by pulsed-wave Doppler from suprasternal and subcostal views, respectively. The duration and end-diastolic velocity of reversed flow were measured. Diastolic flow reversal was subsequently categorized into none-brief, intermediate or holodiastolicprominent [18] Quantitative Doppler were the result of subtraction of the second from the first observation. For inter-observer comparisons, the differences were the result of the subtraction of the average observation (ȳ j ) from the individual observation (y ij ). Differences among observers were plotted using the method proposed by Jones et al. [23] for graphical assessment of agreement with the mean between multiple observers. In this method; d ij = y ij − ȳ j (y-axis) is plotted against y j (x-axis) where y refers to the measurements, ȳ refers to the mean measurement, i refers to observers and j refers to subjects (so ȳ j is the mean of the measurements for subject j). The 95 % limits of agreement (95 % LOA) with the mean are estimated as ±1.96 × s, where s is an estimate of the SD of interobserver differences (for the four observers) and is calculated as the square root of the variance of differences.
Inter-observer and intra-observer agreement on categorical parameters and inter-observer agreement on the PVL grade were expressed as kappa coefficient (κ).
Statistical analysis was performed with SPSS 23 (IBM, Armonk, NY, USA). All probability values were two-tailed, and a p value <0.05 was considered significant.
Based on the reproducibility of the individual parameters, the grading scheme (Table 4) was set up and combined six qualitative and three semiquantitative reproducible parameters. Table S2 shows the number of patients in each of the PVL grades as defined by the four observers using this scheme. Inter-observer grade agreement was achieved in 86 % of cases with a kappa coefficient of 0.79 ( Table 5).
Echocardiography (EAE) for the evaluation of native [18]/ prosthetic [19] aortic regurgitation (AR). Regurgitation volume was calculated as the difference between the stroke volumes at the left and right ventricular outflow tracts, derived from left ventricular outflow tract diameter (LVOTd) and velocity time integral (VTI LVOT ) and right ventricular outflow tract diameter (RVOTd) and VTI (VTI RVOT ).
In the second phase, data on the inter-and intra-observer reproducibility of the individual parameters were used to generate a reproducible PVL grading scheme. Parameters with the best inter-and intra-observer agreement and the least variability were chosen.
In the third phase, PVL severity was graded by the four observers in the 50 echocardiograms using the tailored scheme. The latter combined several qualitative and semiquantitative parameters of PVL severity. The qualitative features were initially used to categorize patients into clear none-trace PVL, clear severe PVL or an intermediate category. In cases in the intermediate zone, we used three semiquantitative parameters to allocate patients into one of four "granular" [20] sub-classes; mild, mild-to-moderate, moderate, and moderate-to-severe. The latter were then collapsed into two classes (mild and moderate) yielding a 4-class (none-trace, mild, moderate, and severe) final scale. We used the cut-points defined by the ASE/EAE guidelines [18], and experts' consensus [21] and opinion [22]. In the first 15 studies, independent assessment by the four observers was routinely followed by a consensus grading to align the interpretation of qualitative parameters. More than 1-class disagreement (across the 6 subclasses) in the independent assessments occurred only in two cases. Those 15 cases were subsequently excluded from statistical analysis which was confined to 35 independently-adjudicated cases. Echocardiographic studies varied in image quality but were adequate for grading of PVL (no transvalvular regurgitation was observed), using at least two parameters of severity.

Statistical analysis
Continuous variables are summarized as mean ± standard deviation (SD) and categorical variables as frequency/ percentage of the studied group. Intra-and inter-observer agreement of numerical parameters was expressed as intraclass correlation coefficient (ICC). For inter-observer ICC, pairwise comparisons of the four observers (6 comparisons) were averaged. The p value for the averaged ICC was determined according to the degree of freedom (number of pairs). Intra and inter-observer variability was expressed as a coefficient of variation (CV) calculated as the SD of inter-/intra-observer difference divided by the population mean. For intra-observer rereads, differences   Table 3 Indices of intraobserver variability and agreement for parameters of PVL severity circumferential extent vs. a multiparametric multi-window approach) and schemes (condensed vs. granular classification), interobserver PVL grade agreement (39-61 %) and weighted kappa estimates (0.48-0.52) were modest [20].

Kappa coefficient
Our approach was to first investigate the reproducibility of the individual parameters to set-up a scheme that combines the most reproducible ones. To improve practicality, quick qualitative features were primarily used to broadly categorize patients. Afterwards, reproducible semiquantitative parameters were applied in a granular manner. The latter concept (granular classification) was previously shown to improve reproducibility of PVL grading and can easily be collapsed into the ordinary 4-class scheme [20]. The latter is more familiar to the clinicians to interpret and more aligned with other techniques (e.g. angiography and magnetic resonance imaging). This approach resulted in an inter-observer agreement on the PVL grade in 86 % of cases, 0 % greater than 1-grade disagreement and a kappa statistic of 0.79, denoting an excellent reproducibility [25].
Color and continuous-wave Doppler parameters showed favorable reproducibility, while aortic flow and quantitative Doppler parameters were less reproducible. Altiok et al. [26] reported intra-and inter-observer variability of

Discussion
The main findings of the present study are that: (1) color Doppler and continuous wave Doppler parameters are more reproducible than other parameters of PVL severity, especially those entailing complex computations (quantitative Doppler); and that (2) a simplified 2-step granular scheme combining reproducible qualitative and semiquantitative parameters improves the inter-observer reproducibility of PVL grading.
The reported rates of PVL in different TAVI trials and registries ranged from 40 to 67 % for trivial to mild and from 7 to 27 % for moderate to severe AR [14,24]. In recently published data from a large series treated with a balloon-expandable valve, the incidence of moderate-severe PVL was reported to be 27 % [24]; more than twofolds the incidence reported in former clinical trials utilizing the same valve technology [2]. Those discrepancies are largely to blame on the low reproducibility of the currently used methods to quantitate PVL.
In a random sample from the PARTNER trial, a highlyconfident grading of PVL was possible in 62 % of studies, while it was low/uninterpretable in 13 % [20]. In spite of applying different approaches (one that heavily weighs jet   supporting the use of CWD are worth-discussing. First; is that available data supports the correlation between the invasively measured transvalvular diastolic pressure gradient and patients' outcomes [28]. Second; is that an index that accounts for the hemodynamics on either side of the aortic valve (stiff aorta and small stiff ventricle) should more accurately reflect the hemodynamic significance of an AR jet. It is therefore more relevant to set-up TAVI-specific cut-points of pressure half time as a hemodynamic index of AR severity than precluding its use. Specific cut-points of severity (reflecting the different hemodynamics of PVL from chronic native AR) were thus adopted in the present analysis, but are yet to be further validated.
An interesting counterintuitive finding of the present analysis is that intra-observer agreement and variability were too close to the inter-observer comparisons for most parameters. Similarly, results were quite similar for the four observers despite the wide range of experience. Both findings indicate that the variability reported here is inherent to the parameters of interest with minimal influence of the setting of analysis.

Limitations
All included echocardiographic studies involved a selfexpanding transcatheter aortic valve. Although applicable 73.5 ± 52.2 and 108 ± 64.7 % for regurgitation volume and 75.2 ± 55.9 and 120.3 ± 62.3 % for regurgitation fraction of post-TAVI PVL. Noteworthy, in the present study, the component basic measurements of quantitative Doppler criteria showed good reproducibility. Variability, however, overinflated as imputations were applied and increased as imputations were more complex (Fig. 2).
It is widely believed, with little supportive evidence, that the hemodynamics of post-TAVI AR are different from that of chronic native AR [27]. Accordingly, the use of Doppler parameters sensitive to hemodynamics (including CWD parameters) in the assessment of post-TAVI AR is subject to experts' criticism. On the other hand, two arguments  Table 4 The final PVL grading scheme set-up after considering the reproducibility of the individual parameters *Two positions in each of the three long-axis views (PLAX, A5C and A3C). Abbreviations as in Table 1 to other valve types, the findings should be generalized with caution. The cut-points used in classifying the severity of PVL are inadequately validated in TAVI patients [21]. Accuracy of those parameters is, however, beyond the scope of the present study.

Conclusion
Reproducibility of PVL assessment by transthoracic echocardiography can be improved by using a simplified approach combining reproducible color and continuous wave Doppler parameters.