Quality Improvement Initiative to Improve Endoscopic Reassessment of Eosinophilic Esophagitis
Drs Khalaf and Rosenwald share the first coauthorship to the article.
This work is partly supported by a Cystic Fibrosis Foundational Grant Award no. Khalaf17B0 and National Institutes of Health Training Grant 5T32-DK067009-12 to R.K., Redcap NIH/NCRR Colorado CTSI Grant Number UL1 RR025780 to K.R. and R.K., and Sondheimer Fellow Funds from Children's Hospital Colorado.
The authors report no conflicts of interest.
Abstract
Background:
The eosinophilic esophagitis (EoE) endoscopic reference score (EREFS) is a validated system for description, recognition, and reporting of EoE findings during esophagogastroduodenoscopy (EGD). This scoring system correlates with esophageal eosinophilia and therapeutic responses and has validated diagnostic accuracy with good inter- and intraobserver reliability in pediatric and adult patients. In this study, we aimed to improve physician education on and documentation of EREFS and correlate EREFS scoring with eosinophil density on histology.
Methods:
Applying the “Plan, Do, Study, Act” methodology for quality improvement between October 2018 and November 2019, we established a baseline rate of EREFS completion by review of the electronic medical record (EMR). Key drivers were identified, and 3 interventions were implemented.
Results:
Over 12 months, 542 distinct endoscopies were performed on 410 patients for EoE surveillance. Patients were 68% male with a mean age of 10.9 years (SD 5.7 years), mean EREFS score of 2.14 (SD 1.88), and mean peak eosinophil count 30.9 eos/hpf (SD 37.1 eos/hpf). Baseline EREFS completion rate of 72.7% (90% CI, 67.4-77.4). Following all 3 PDSA cycles, EREFS completion rate significantly improved to greater than desired target of 90% (94.9%; 90% CI, 90.6-97.6; P < 0.001).
Conclusion:
Interventions including provider education and the inclusion of EREFS in documentation templates can increase adoption rates of EREFS among providers caring for patients with known EoE.
What Is Known
- The Eosinophilic Esophagitis (EoE) Endoscopic Reference Score (EREFS) classification is validated for description, recognition, and reporting of gross endoscopic findings suggestive of EoE during endoscopy.
- EREFS has demonstrated strong correlation with esophageal eosinophilia, histopathologic scoring systems, and treatment response in adults and children with EoE.
What Is New
- EREFS is inherently accurate even when implemented among endoscopists of varying practice patterns and diverse clinical skillsets, as demonstrated by a significant correlation between peak eosinophil count and EREFS total and component scoring.
- Providers achieved high, meaningful, and sustainable EREFS scoring rates with brief education and the inclusion of EREFS in documentation templates.
INTRODUCTION
Eosinophilic esophagitis (EoE) is a chronic, immune-, and antigen-mediated inflammatory condition of the esophagus, generally characterized by symptoms of vomiting, food refusal, and weight loss in young children and dysphagia and esophageal food impaction in older children, adolescents, and adults (1). Diagnosis of EoE is clinicopathologic, requiring compatible symptoms of esophageal dysfunction and esophageal histopathology with ≥15 eosinophils per high power field (Eos/hpf), not attributable to non-EoE disorders (2).
While not part of the diagnostic criteria, visual endoscopic findings during esophagogastroduodenoscopy (EGD) may serve as a valuable adjunctive measure to symptoms and histology in assessing response to therapeutic intervention. The EoE Endoscopic Reference Score (EREFS) classification is a validated system for description, recognition, and reporting of gross endoscopic findings suggestive of EoE findings during EGD (3). EREFS provides a score from 0 to 9 based on the presence or severity of 5 component scores: exudates, rings, edema, furrows, and strictures. This scoring system has validated diagnostic accuracy with good inter- and intraobserver reliability for component scores and total score (3, 4). Furthermore, EREFS score has demonstrated strong correlation with esophageal eosinophilia (5-8), histopathologic scoring systems for EoE (6), and treatment response (5-8) in adults and children.
Implementation of EREFS as a part of routine clinical practice helps to standardize care across sites and endoscopists, across repeat endoscopies throughout the course of diagnosis and treatment, and for use in therapeutic studies.
This quality improvement project was motivated by the observation of insufficient EREFS completion rate to ensure the vast majority of EoE patients would have endoscopic as well as symptomatic and histologic parameters to evaluate their treatment response. Specific improvement was desired among known EoE patients undergoing repeat EGD. From October 2018 to May 2019, the Digestive Health Institute at Children's Hospital Colorado performed 170 repeat upper endoscopies for patients with known EoE in the Main Campus Procedure Center, with a total EREFS completion rate of 72.7%.
Through this initiative, our primary aim was to increase the rate of completion of EREFS to greater than 90% among EoE patients undergoing repeat EGD. As a secondary aim, we sought to understand the correlation of the EREFS system with markers of EoE disease activity in our population. We hypothesized that improving ease of data entry and physician education would improve EREFS scoring and that EREFS would correlate with eosinophil density on histology.
As a balancing measure, we sought to observe whether broadening acceptance of EREFS among endoscopists would have an effect on the correlation of the EREFS system with disease activity, in particular esophageal eosinophilia.
METHODS
Context
This quality improvement initiative was implemented in the Digestive Health Institute (DHI) at Children's Hospital Colorado main campus and was exempt from institutional review board review. Endoscopies ordered by EoE specialists may be performed by any gastroenterologist at the institution, not necessarily the ordering provider or an EoE specialist. Pediatric gastroenterology fellows may or may not participate in the procedure and complete its associated documentation.
Interventions
We utilized the Plan, Do, Study, Act methodology for quality improvement for the study period from October 2018 through February 2020. Inclusion criteria for study were any procedures performed in the Main Campus Procedure Center for individuals with prior diagnosis of EoE. For the purposes of this study, prior diagnosis of EoE was defined as any one of the following: (a) EoE in Problem List before procedure, (b) EoE listed as indication for procedure in the procedural history and physical, or (c) previous histopathology with >15 eosinophils per high power field (eos/hpf) AND documentation (e.g., prior clinic visits or telephone encounter) that delivers diagnosis of EoE.
We first established baseline rate of EREFS completion by review of the electronic medical record (EMR). Investigators performed a rolling chart review of all endoscopic procedures to determine whether inclusion criteria were met in each case. For included cases, investigators collected data regarding demographics, procedure type, faculty ± fellow performing procedure, EREFS total score and subscores, biopsy locations, and peak esophageal eosinophilia on histology.
Key drivers were identified for incomplete EREFS completion in discussion with endoscopists (faculty and fellows), EoE specialists, and endoscopy center leadership. Particular key drivers felt most likely to warrant intervention included lack of training on EREFS completion, lack of education regarding diagnostic and prognostic value of EREFS, lack of standardized or automated documentation in the electronic medical record, and lack of access to visual reference for quickly and accurately performing EREFS. Three distinct interventions were subsequently implemented. First, in June 2019, we added an automatic drop-down EREFS menu to the template for EGD documentation in the EMR. Second, in August 2019, we provided an in-person educational session for faculty and fellows regarding how to complete EREFS and the value of EREFS for clinical care. The third intervention was 3-fold and was implemented in November 2019. This included (a) addition of an automatic drop-down EREFS menu to the procedure documentation templates for both EGD + esophageal dilation and EGD + colonoscopy, (b) placement of a laminated EREFS visual atlas or grading guide adjacent to provider workstations in the procedure center, and (c) an in-person educational session for faculty only at the monthly departmental faculty meeting. EREFS completion rates were plotted on a run chart monthly pre- and postintervention over the study period.
Statistical Analysis
Descriptive analyses (mean, standard deviation, median, interquartile range [IQR], and frequency distributions) were used to describe patient demographics over entire study period and grouped by intervention periods. Fisher exact test and chi-square test were performed as appropriate to assess the change in EREFS completion rate across study periods and its associations with endoscopy procedure types and completion rate. Spearman's rank correlation coefficients were calculated for assessing peak eos/hpf, and EREFS total score was tested. Segmental linear regression of monthly EREFS rates was used to delineate the temporal trend over each study period. Nonparametric ANOVA was performed to test the difference in peak eos/hpf between categories of each EREFS component. All statistical tests were 2-tailed, and significance level was set at 0.05. Ninety percent of confidence intervals of EREFS completion rates were reported for making one-tailed inference. Because of nonnormality of features, Kruskal-Wallis test was used for overall test, and Wilcoxon rank-sum test for pair-wise comparison. No adjustment of P values was used for multiple comparisons. Statistical analyses were performed using SAS version 9.4 (SAS Institute Inc., Cary, NC).
RESULTS
Over 12 months, 542 distinct endoscopies were performed on 410 patients for an EoE indication. Sixty-eight percent of patients were male with a mean age of 10.9 years (SD 5.7 years), mean EREFS score of 2.14 (SD 1.88), mean peak eosinophil count 30.9 eos/hpf (SD 37.1 eos/hpf) (Table 1). Seventy-one percent of cases were EGD only, 18% EGD with esophageal dilation, and 9% EGD with colonoscopy. Faculty performed 74% of procedures alone and 26% with a gastroenterology fellow. Biopsies were obtained from the proximal and distal esophagus in 98% of procedures.
Target EREFS Completion Rate Sustained Following All Interventions
During the preintervention phase of 8 months, 234 cases met inclusion criteria, 170 of which had EREFS documented yielding a baseline EREFS completion rate of 72.7% (90% CI, 67.4-77.4) (Fig. 1). Fifty-four out of 67 cases (80.6%; 90% CI, 70.9-88.1) had EREFS completed following the first intervention and 82 out of 104 cases (78.9%; 90% CI, 71.2-85.2) following the second intervention. Following all 3 PDSA cycles, EREFS completion rate was significantly improved from baseline and was completed in 130 out of 137 cases (94.9%; 90% CI, 90.6-97.6; P < 0.001), hence achieving project target of greater than 90% completion rate.
Comparison of completion rates pre- and post-all interventions and by procedure type. A) Comparison pre- and post-all interventions for all procedures compiled together. B) Comparison between phases of the quality intervention project. For all procedure types compiled together. Fisher's exact test and Chi-square test show that this quality improvement project achieved its target of 90% completion of EREFS (mean 4.9; 90% CI, 90.6-97.6). C) Comparison of pre- and post-all interventions divided by procedure type. Note, the sample sizes are 153, 19, and 33, respectively, for upper endoscopy, upper endoscopy/colonoscopy, and upper endoscopy with esophageal dilation. The P values are <0.0001, 0.09, and 0.05, respectively, by Fisher's exact test.
Figure 2 shows the crude monthly rates for all procedures, for those performed by both faculty and fellows, and for those performed by faculty alone.
EREFS completion rate versus time by 3 groups (90% CI). Completion rates for cases completed by (A) trainees in conjunction with faculty, (B) faculty alone, and (C) either trainees with faculty or faculty alone (i.e., all procedures). In these plots, the blue line is the fitted linear regression line. The dots represent the monthly completion rate, and the vertical lines are the 90% CI for the completion rates estimated using exact method. Segment or phase-specific slopes were not significant for each segment in each plot. However, we saw that there is an increasing trend as soon as the first intervention and improvement in rate of ERFS completion after all interventions were sustained.
When cases are further stratified by procedure type, rate of EREFS completion preinterventions was 66.0% (90% CI, 53.9-76.8) for EGD, 66.7% (90% CI, 34.5-90.2) for EGD with colonoscopy, and 66.7% (90% CI, 34.5-90.2) for EGD with esophageal dilation. Following all interventions, rate of EREFS completion increased to 94% (90% CI, 88.5-97.4; P < 0.0001) for EGD, 100% (90% CI, 74.1-100; P = 0.09) for EGD with colonoscopy, and 95.8% (90% CI, 81.7-99.8; P = 0.05) for EGD with esophageal dilation.
Correlation Between EREFS Score and Eosinophil Count
In this cohort, increased peak eosinophils per high power field were significantly associated with a higher EREFS score in the total cohort (Spearman ρ = 0.061, P < 0.001) (Fig. 3). This correlation was noted pre- and postinterventions (Fig. 3B, C). We further compared the peak eosinophil count per high power field across categories of each of the 5 components of EREFS scores (Fig. 4). Peak eosinophilic per hpf was significantly increased with a positive score for any individual EREFS component compared with an individual score of 0.
Correlation of EREFS score and peak eosinophilic count for (A) the complete cohort, (B) the cohort divided depending on the time of procedure completion either pre- and postinterventions, and (C) the cohort divided based on the time of procedure completion either pre, between, or post-all interventions.
Peak eosinophil count across categories of each of the 5 components of EREFS. A) peak eosinophils versus edema score, (B) peak eosinophils versus ring score, (C) peak eosinophils versus exudate score, (D) peak eosinophils versus furrows score, and (E) peak eosinophils versus stricture score.
DISCUSSION
Despite its characterization in 2013 by Hirano et al (3), EREFS scoring continues to remain underused. In a recent nationwide survey of 1393 adult gastroenterologists in Germany with a response rate of 29.6%, the EREFS score was mostly either unknown (44.3%) or not routinely used (52.2%) (9). Our quality improvement initiative aimed at increasing EREFS documentation rate provides several interesting findings. First, with targeted interventions based on key drivers for poor EREFS completion, we were able to achieve and sustain a high-target EREFS completion rate (>90%) among a large cohort of attending and trainee physicians at a pediatric tertiary care center while maintaining score correlation with peak eosinophil count. Second, with brief education, EREFS scoring appears to be readily accepted and inherently accurate even when implemented among endoscopists of varying practice patterns and diverse clinical skillsets, as demonstrated by a significant correlation between peak eosinophil count and EREFS total and component scoring, both before and after efforts to expand implementation among this large physician cohort. Taken together, our results indicate that with appropriate education and target intervention, providers can achieve high and meaningful EREFS scoring rates that are sustainable.
Previous work has shown that EREFS scoring system accurately identifies disease activity in children with and without visible disease and across the broad age spectrum of pediatrics. In a longitudinal cohort study of children 2 to 17 years of age undergoing repeat endoscopy for EoE, EREFS accurately identified disease activity in children, measured by peak eos/hpf (diagnostic EREFS: R = 0.82, posttreatment EREFS: R = 0.59) (5), similar to the posttreatment EREFS correlation demonstrated in this study (Pearson R = 0.57 [95% 0.50-0.63] and Spearman ρ = 0.61 [0.54-0.66]). In addition, a recent retrospective study, of 878 EoE cases from 2002 to 2018 by Eluri et al, reported 101 (11.5%) individuals with documentation of endoscopically normal esophagus but further stratified this cohort to indicate that the proportion of individuals with a normal esophagus decreased from 21% before the first EoE guidelines to 7% (P < 0.01) after introduction of EREFS. This finding further highlights that systematically evaluating the visual appearance of the esophagus leads to improved detection and recognition of endoscopic findings of EoE (10). Furthermore, Ahuja et al compared EREFS use in younger (⩽10 years) and older (>10 years) pediatric patients in a cohort of 99 individuals and found that EREFS scoring had similar specificities (0.88 versus 0.89) and positive predictive values (0.89 versus 0.91) in both age groups (11).
Variability of EREFS scores has been reported in the detection of active and inactive EoE. Hiremath et al studied 189 paired EREFSs, EoE histologic scoring system, and peak eosinophil count to develop a model of disease activity and determined that the relationship between total endoscopic and histological scoring is stronger in active versus inactive EoE (r = 0.41 versus 0.24; P = 0.09). Compared with EREFS, histological scoring had a significantly higher area under the curve (0.78 versus 0.92; P = 0.04) to predict active EoE; thus, the authors concluded that EREFS scoring alone is not a reliable marker of tissue involvement in EoE and should be paired with histologic findings to better predict disease activity (6). Yet, in a systematic review aimed at identifying scoring indices used for the measurement of disease activity in EoE, appraising their operating properties, and discussing their value as outcome measures, authors found 130 studies eligible for the review and concluded based on data from these that EREFS scores are the most reliable and responsive endoscopy measure of disease activity (12).
Our study has several limitations. Our study was conducted at a large tertiary care center with a dedicated team of pediatric gastroenterologists with expertise in the care of pediatric patients with EoE, which may have increased the likelihood of providers at this institution to adopt and use the EREFS scoring tool. Furthermore, this study involved trainee physicians who showed earlier adoption of the scoring system and who may not be involved in the care of similar patients elsewhere.
Overall, our results suggest that interventions including provider education and the inclusion of EREFS in documentation templates can increase adoption rates of EREFS among providers caring for patients with known EoE. These findings have important implications in the management of patients with EoE, as changes in endoscopic appearance provide an additional outcome measure of clinical response to treatment and may help in clinical care decisions when used in conjunction with histologic findings. Ongoing PDSA cycles will assess for the durability of response.
ACKNOWLEDGMENTS
The authors would like to thank Drs. Glenn Furuta and Ikuo Hirano for their thoughtful input in early stages of project development.
