Patient Uncertainty as a Predictor of 30-day Return Emergency Department Visits: An Observational Study
Abstract
Objective
The objective was to examine the relationship between patient uncertainty at the time of emergency department (ED) discharge as measured by the “Uncertainty Scale” (U-Scale) and 30-day return ED visits. We hypothesized that a higher score on the U-Scale predicts a higher likelihood of a 30-day return ED visit.
Methods
This was a cross-sectional single-site pilot study performed with adult patients discharged from an urban academic ED to assess the relationship of U-Scale total and subscale scores with 30-day return ED visits. We collected demographic and U-Scale scores at the time of ED discharge and subsequent 30-day ED utilization data by follow-up telephone call.
Results
No association was found between the total U-Scale score and subsequent ED utilization. Patients with higher uncertainty on the Treatment Quality subscale of the U-Scale had higher odds of a 30-day return ED visit (adjusted odds ratio [AOR] = 1.16), while patients with lower uncertainty on the Decision to Seek Care subscale had higher odds of a 30-day return ED visit (AOR = 0.68).
Conclusion
Patient uncertainty as measured by the U-Scale total score was not predictive of subsequent ED utilization. However, uncertainty related to treatment quality and the decision to seek care as measured by the U-Scale subscales may be important in predicting repeat ED utilization. Unlike individual patient factors such as age and race that have been associated with frequent ED visits in prior studies, these domains of uncertainty are potentially modifiable. Providers and health systems may successfully prevent recurrent acute care encounters through implementation of interventions designed to address patient uncertainty. Further work is needed to refine the U-Scale and test its predictive utility among a larger patient cohort.
Background
As health care reimbursement shifts toward value over volume, it is critical for health systems to anticipate the needs of patients and facilitate care delivery in a patient-centered, high-value, and lower-cost manner. Much attention has been dedicated to efforts to reduce acute care utilization, a commonly cited driver of high cost care.1 Prior research focused on identifying factors that predict frequent emergency department (ED) use has found numerous patient-level factors associated with higher frequency of ED visits including sex, race, mental health status, employment status, Medicaid and insurance status, arrival mode to the ED, and prior ED utilization.2-4 Illness severity and comorbidity indices have also been shown to predict risk of ED use and hospital readmissions.5-8 Yet these factors provide an incomplete picture of patients at high-risk of ED utilization. Studies that measure individual-level predictors easily available in administrative data such as sex or race tend to capture only a small amount of the variability in ED usage or to not report total variability explained.2, 9-11 Further, there is evidence that stable personality traits account for a large proportion of variability.12 Patient-level factors that could be addressed through provider and system interventions are needed.
Importance
Our prior work suggests that patients’ decisions regarding when and where to seek care are influenced by potentially modifiable factors, such as emotional state, health system trust, and satisfaction. Patients report that they seek care in the ED because they are afraid and uncertain about various aspects of their symptoms13-15 and that they return to the ED because of ongoing uncertainty related to their symptoms.16 Uncertainty is defined as “the inability to determine the meaning of illness-related events.”17 Mishel18 developed the Uncertainty in Illness Scale to measure the level of uncertainty in hospitalized patients with chronic conditions. The uncertainty stress scale was then developed to measure uncertainty during illness as well as the stress, threat, and positive feelings associated with the state of uncertainty.19 These scales do not address the unique aspects of unscheduled acute care pointing to the need to develop a scale to measure uncertainty during an episode of acute care. Understanding the relationship between patient uncertainty and acute care utilization is essential so that health systems can better predict risk of acute care utilization, design interventions to meet patient needs, and more precisely deploy resources to improve health care outcomes.
Goals of This Investigation
Our team developed a scale to measure patient uncertainty related to symptoms, the “Uncertainty Scale” (U-Scale), which demonstrated evidence of content validity, internal consistency, reliability, and concurrent validity in initial testing.20 Exploratory factor analysis during scale testing and validation suggested the U-Scale is composed of seven subscales. The goal of this pilot study is to assess the relationship between patient uncertainty at the time of ED discharge as captured by the U-Scale total and subscale scores and subsequent ED utilization. We hypothesized that a higher total score on the U-Scale predicts a higher likelihood of a 30-day return ED visit. Analyses conducted with the subscale scores were exploratory models to investigate whether certain domains of uncertainty are individually related to subsequent ED utilization.
Methods
Study Design and Setting
This cross-sectional, single-site pilot study was conducted to assess whether higher patient uncertainty at the end of an ED visit as measured by the U-Scale is predictive of subsequent ED use. Data for this study were collected at the same time as data for scale reliability and validity testing of the U-Scale.20 Participants were recruited from a single ED at Thomas Jefferson University Hospital, a large academic hospital located in an urban center. This observational study is compliant with STROBE guidelines.
Selection of Participants
Trained research assistants screened the electronic medical record to identify potentially eligible patients and then approached patients to complete an eligibility assessment. English-speaking adult (18 years and older) patients who were being discharged from the ED were recruited at the end of their ED visit. Patients with communication barriers (including hearing and visual impairments) or severe medical acuity were excluded from the study. Patients who were unable to provide consent due to intoxication, mental impairment, or altered mental status; who were undergoing medical clearance or in police custody; or who had already been enrolled in the study during a previous ED visit were also excluded from the study. Finally, pregnant women were excluded since their health status gives them a unique profile of needs and perspectives on their symptoms that are potentially not generalizable across the broader population of ED patients. The study received institutional review board approval, and written informed consent was collected from all patients at the time of enrollment.
Measurements
Trained research assistants recorded and stored all study measurements using a REDCap database. Participant characteristics and U-Scale scores were obtained during the enrollment ED visit. Participants were contacted by telephone 30 days after study enrollment to provide self-report data of the number of times they visited an ED in the 30 days after their initial enrollment ED visit.
Participant Characteristics
Participant information was collected via self-report at enrollment and included age, sex, household income category, level of education, race/ethnicity, insurance status, whether they had a primary care provider, presence of a chronic condition (cardiovascular disease, cancer, pulmonary disease, endocrine disorder, kidney disease, mental health disorder, nervous system disease, chronic infectious disease, gastrointestinal disease, and chronic pain), and whether they had an ED visit in the 30 days prior to study enrollment. We collapsed the conditions into five categories (cardiovascular, pulmonary, endocrine, mental health, and gastrointestinal) and then created a categorical variable for chronic disease categories: 0, 1–2, and 3 or more. These variables were used to characterize the overall sample and to identify potential covariates for the primary analyses.
U-Scale
The U-Scale is a 30-item scale that captures domains of patient uncertainty related to the experience of symptoms. Individual items are responded to on a 1 to 5 Likert scale; thus, the nominal range for the total U-Scale score is 30 to 150, with higher scores indicating higher uncertainty. The scale demonstrated evidence of content validity, internal consistency, reliability, and concurrent validity in initial psychometric testing.20 Exploratory factor analysis suggested seven nonoverlapping subscales capturing subdomains of patient uncertainty: Treatment Quality, Self-management, Diagnosis, Worries and Concerns, Decision to Seek Care, Self-efficacy, and Psychosocial Concerns. Descriptions, example items, and reliability for the subscales are shown in Table 1.
| Subscale Name | Subscale Description | Item Example | Number of Items | Internal Consistency Reliabilitya |
|---|---|---|---|---|
| Treatment Quality | Perceived quality of care received | I often feel like my doctors don't give me enough information about my test results | 9 | 0.86 |
| Diagnosis | Desire for explanation about cause and treatment for symptoms | I don't have an explanation for what is causing my symptoms | 8 | 0.82 |
| Self-management | Knowledge and ability to manage one's own symptoms | I have the knowledge and ability to treat my symptoms | 4 | 0.71 |
| Worries & Concern | Emotional distress related to symptoms | Feeling nervous about my symptoms is causing emotional distress | 3 | 0.66 |
| Decision to Seek Care | Ability to determine when to seek care | I don't know which symptoms I should go see a medical professional about | 2 | 0.76 |
| Self-efficacy | Ability to identify how and where to get help when needed | I don't know where to go for help when experiencing different kinds of symptoms | 2 | 0.61 |
| Psychosocial Issues | Concern about how medical needs will impact life | I worry that seeking medical care will get in the way of my commitments at home or work | 2 | 0.60 |
- U-Scale = Uncertainty Scale.
- a Cronbach's alpha
30-day ED Utilization
The primary outcome was coded dichotomously as presence/absence of at least one return ED visit. As this study was conducted in an urban metropolitan area where patients have access to a number of different ED and health systems, 30-day ED utilization was collected by patient self-report to capture ED use at any health system.
Data Analysis
Our analysis approach was designed to assess the relationship between uncertainty, including subdomains of uncertainty, as measured by the U-Scale total and subscale scores and the occurrence of at least one 30-day return ED visit. We first calculated descriptive statistics for the sample, including patient descriptive variables, U-Scale total score, and the seven U-Scale subscale scores. The U-Scale total score and subscale scores were screened for distributional assumptions through visual inspection and calculation of the ratio of the skew and kurtosis values to their standard errors. We then screened the variables at this step for the presence of outliers and influential cases relative to the logistic regression analysis by obtaining the Pearson residuals and leverage values for the U-Scale total and each subscale predictor individually with respect to each case's standing on the outcome. One case was removed based on its standing on the Self-efficacy subscale and its leverage values. The predictor variables with this case removed were all normally distributed, and no more influential cases were identified. Sample sizes for the first and second logistic regression models were 156 and 155, respectively.
We compared these variables between the groups with and without a repeat ED visit within 30 days using chi-square for categorical variables and t-tests for continuous variables. We used an entry criterion of p < 0.10 for variable inclusion in the logistic regression models. With the exception of self-reported prior ED usage in the 30 days before study enrollment, none of the measured patient variables were related to the outcome of 30-day return ED visit at the p < 0.10 threshold. Because participants’ ED usage in the 30 days prior to enrollment was significantly related to the outcome and was also associated with the U-Scale total score and with two of the seven subscale scores, it was included as a covariate in the logistic regression models.
We then estimated two logistic regression models to explore associations between the binary outcome of 30-day return ED visit and the uncertainty variables. Both models used a direct entry method, with the identified covariate of any prior 30-day ED visit entered in the first block. Model 1 used the U-Scale total score in the second block, and model 2 used the seven U-Scale subscale scores in the second block. These models assess the independent contribution of the uncertainty variables, controlling for the identified covariate. The model with the seven U-Scale subscales entered together assesses the association between each subscale and the outcome, controlling for the other subscales, and therefore estimates the effect of each domain of uncertainty allowing for uncertainty in the other domains. We report model fit statistics using the significance of the Hosmer-Lemeshow test21 and percent of correct classifications; Nagelkerke R2 for proportion of variability in the outcome explained by the model; and adjusted odds ratios (AORs), their significance values and confidence intervals (CIs) for regression coefficients. We set an alpha of 0.05 for this study.
As this was pilot work, we did not have any a priori expectations about the magnitude of potential effects of covariables; additionally, the available sample size was constrained as these data were collected for another purpose. Therefore, no a priori power analysis was conducted. For the effects that were significant, we evaluated effect size using a general linear model approach to obtain covariate-adjusted estimates of the between-group differences (for those with and without a 30-day return ED visit) for the U-Scale total and for both subscales that were significantly related to the binary outcome in the second logistic regression model. These analyses controlled for the presence of a prior 30-day ED visit, and the estimates for the subscales also controlled for each of the other subscales. The adjusted between-group difference in each case was divided by the pooled standard deviation (SD) to obtain Cohen's ‘d’—the standardized between-group difference—as an effect size. This effect size reflects the distance between group means in SD units and thus is a scale-free measure of effect for continuous variables.
Results
Characteristics of Study Subjects
We enrolled 200 patients at the time of ED discharge between February 2017 and April 2017 and successfully contacted 156 of the 200 enrolled patients (78%) by telephone for a 30-day follow-up assessment within a mean of 32 days (range, 29–85 days) of their enrollment ED visit. Patients unable to be contacted at follow-up did not differ significantly from initial study sample on any of the demographic characteristics collected including age, sex, income, race, ethnicity, education, insurance status, presence of a primary care provider, or number of chronic conditions, nor did the patients lost to follow up differ on the U-Scale total or subscale scores (results not shown). Descriptive variables and between-group differences for the study population are shown in Tables 2 and 3.
| Characteristic | Whole Sample (N = 156) | With Return ED Visit (n = 22) |
|---|---|---|
| Age (years) | ||
| 18–29 | 53 (34) | 8 (36) |
| 30–39 | 35 (22) | 5 (23) |
| 40–49 | 25 (16) | 2 (9) |
| 50–59 | 29 (19) | 5 (23) |
| 60–69 | 9 (6) | 2 (9) |
| 70+ | 5 (3) | 0 (0) |
| Sex | ||
| Female | 98 (63) | 14 (64) |
| Male | 55 (35) | 8 (36) |
| Something else | 3 (2) | 0 (0) |
| Income | ||
| <$10,000 | 18 (12) | 5 (23) |
| $10,000–$24,999 | 21 (14) | 2 (9) |
| $25,000–549,999 | 36 (23) | 4 (18) |
| >$50,000 | 43 (27) | 4 (18 |
| Missing | 38 (24) | 7 (32) |
| Race | ||
| African American/black | 90 (63) | 14 (64) |
| White | 53 (35) | 7 (32) |
| Something else | 13 (2) | 1 (5) |
| Ethnicity | ||
| Hispanic or Latino | 17 (11) | 1 (4.5) |
| Not Hispanic or Latino | 132 (85) | 20 (91) |
| Decline to answer | 7(4.5) | 1 (4.5) |
| Education | ||
| Less than high school | 15 (10) | 3 (14) |
| High school graduate | 89 (57) | 13 (59) |
| College graduate | 41 (26) | 5 (23) |
| Postgraduate | 10 (6) | 1 (5) |
| Missing | 1 (1) | (0) |
| Has Insurance | ||
| Yes | 149 (96) | 22 (100) |
| No | 7 (5) | 0 (0) |
| Has primary care provider | ||
| Yes | 133 (85) | 19 (86) |
| No | 23 (15) | 3 (14) |
| Missing | 3 (2) | 0 (0) |
| Number of chronic conditions | ||
| 0 | 49 (31) | 7 (32) |
| 1–2 | 93 (60) | 12 (54) |
| ≥3 | 14 (9) | 3 (14) |
| ED visit 30 days priora | ||
| None | 109 (70) | 11 (50) |
| One or more | 47 (30) | 11 (50) |
- Data are reported as n (%).
- a Between-group difference significant, χ2(1) = 4.80, p < 0.05.
| Measure (Range) | With Return ED Visit (n = 22 for Total Score, n = 21 for Subscales) | Without Return ED Visit (n = 134) | Mean Difference (95% CI) (covariate adjusteda) |
|---|---|---|---|
| U-Scale total (34–118) | 74.9 (±17.3) | 72.2 (±18.4) | 2.7 (–0.0 to 6.9) |
| Treatment Quality subscale (13–37) | 23.9 (±5.7) | 21.7 (±4.9) | 2.2 (0.36 to 4.1), d = 0.41 |
| Diagnosis subscale (8–38) | 19.7 (±6.4) | 20.3 (±5.9) | 0.6 (–1.5 to 2.7) |
| Self-management subscale (4–17) | 8.8 (±3.1) | 8.5 (±3.2) | 0.3 (–0.78 to 1.6) |
| Worries & Concerns subscale (3–15) | 8.4 (±3.0) | 8.2 (±2.8) | 0.2 (–0.96 to 1.4) |
| Psychosocial subscale (2–10) | 4.1 (±2.0) | 4.7 (±1.4) | 0.5 (–0.2 to 1.4) |
| Decision to Seek Care subscale (2–10) | 4.2 (±2.0) | 5.1 (±1.9) | 0.9 (0.10 to 1.6), d = 0.46 |
| Self-efficacy subscale (2–10) | 4.0 (±1.5) | 3.5 (±1.8) | 0.5 (–1.0 to 0.08) |
- Data are reported as mean (±SD).
- U-Scale = Uncertainty Scale.
- a All estimates are adjusted by self-reported ED visit in previous 30 days. Each subscale also adjusted by the other subscales. For subscales that were significant predictors of the outcome, Cohen's d is also given.
Main Results
Model 1 Result
Model 1 tested prediction of 30-day return ED utilization using prior 30-day ED visit as a covariate in step 1 and with the U-Scale total score entered in step 2. The model chi-square was non-significant (χ2(2) = 14.64, p > 0.10). In this model, only the self-reported prior 30-day ED use was a significant predictor (AOR = 2.6 [95% CI = 1.04–6.70], p < 0.05), indicating a 2.6-times increase in the odds of having a 30-day return ED visit for those who reported a prior 30-day ED visit. The U-Scale total score was not significant (AOR = 1.01 [96% CI = 0.98–1.03], p > 0.10). The nonsignificant Hosmer-Lemeshow test indicated acceptable model fit (p = 0.47).
Model 2 Result
Model 2 tested prediction of 30-day return ED utilization using prior 30-day ED visit as a covariate in step 1 and with the seven U-Scale subscales entered in step 2. The model chi-square was not significant (χ2(8) = 15.38, p = 0.052). Nagelkerke R2 indicated that 17.2% of the variability in the outcome was captured by the model, with 87.1% correct classifications. Hosmer-Lemeshow test indicated acceptable model fit (p = 0.75). In addition to the covariate of previous 30-day ED use, the (Uncertainty about …) Treatment Quality and Decision to Seek Care subscales were significant predictors in the model. Higher uncertainty on the Treatment Quality subscale was associated with increased odds of having a 30-day return ED visit (AOR = 1.16 [95% CI = 1.02–1.32], p < 0.05). Specifically, each one-point increase in the Treatment Quality subscale was associated with a 16% increase in the odds of a return visit. In contrast, higher scores on the Decision to Seek Care subscale were associated with lower odds of having a return ED visit (AOR = 0.68 [95% CI = 0.47–0.98], p < 0.05). Specifically, every one-point increase in the Decisions to Seek Care subscale was associated with a 32% decrease in the odds of a return visit.
Effect Size
For the significant subscales, the Cohen's ‘d’ values were 0.46 for the Treatment Quality subscale and 0.41 for the Decision to Seek Care subscale. These standardized differences are considered medium effects22 and are consistent with the proportion of variance explained in the models; in the logistic regression model containing all the subscales simultaneously, over 17% of the variability in the outcome was explained.
Discussion
Our analyses demonstrated that patient uncertainty as measured by the U-Scale total score was not predictive of recurrent 30-day ED visits; however, exploratory analyses suggested that uncertainty as measured by two of the subscales is predictive of ED return visits. Specifically, patients with higher uncertainty measured on the Treatment Quality (perceived quality of care received) subscale of the U-Scale had higher odds of a 30-day return ED visit, while patients with higher uncertainty on the Decision to Seek Care (ability to determine when to seek care) had lower odds of a 30-day return ED visit. Analysis of effect size for these subscales along with amount of variance explained in these models suggests a moderate relationship between these domains of uncertainty and self-reported ED utilization. There was no association between uncertainty as captured in the other five subscales: Diagnosis (desire for explanation about cause and treatment for symptoms), Worries and Concerns (emotional distress related to symptoms), Self-management (knowledge and ability to manage one's symptoms), Self-efficacy (ability to identify how and where to get help when needed), and Psychosocial Concerns (concern about how medical needs will impact life).
While we did not find a significant relationship between the overall U-Scale score and subsequent ED visits, the relationship between two specific subscales suggest that the relationship between patient uncertainty and ED utilization may be more nuanced than the total scale score captures and that uncertainty related to specific aspects of symptom management—particularly treatment quality and decision making—may be more impactful than other aspects of uncertainty. Prior work has demonstrated that patient perceptions of treatment quality impact a number of important outcomes including patient satisfaction,23-26 choice of provider,27 and adherence to medical advice.28, 29 In addition, trust in medical provider and health care institution impacts patient adherence to recommendations,28, 30 self-rated health,31 clinical outcomes,30, 32 and overall satisfaction with care.33 Our finding that higher uncertainty regarding treatment quality is associated with higher odds of a 30-day return ED visits adds to the literature by suggesting a direct link between perceived quality of care received during health care encounters and subsequent ED utilization. This suggests that understanding and intervening on factors that influence patient perceptions of treatment quality may reduce subsequent resource utilization. A model developed by Sofaer and Firminger34 suggests that patient perceptions of quality of care are influenced by patient expectations and prior experiences. Sofaer and Firminger also note that patients have distinct criteria for quality of care, and these criteria are often implicit. Providers and health care organizations wishing to impact patient utilization may benefit from establishing shared criteria for quality of care that incorporates patient perspectives and is explicitly articulated during the acute care encounter.
A notable and unexpected finding is that patients who scored higher on the Decision to Seek Care uncertainty subscale were less likely to return to the ED—that is, patients who were more uncertain about whether their symptoms required evaluation in the ED were less likely to return to the ED for a subsequent visit within 30 days. These findings may be explained in a few ways. First, patients who are not confident in deciding when to go to the ED are likely to make a different decision regarding whether and when to seek care in the future for ongoing or new symptoms because they do not have routine health care use patterns. Alternatively, patients may have been uncertain only about their most recent decision to seek care, and their ED visit itself may have resulted in them not deciding to seek ED care again in the future. Finally, patients who were unsure about their decision to seek care may have had symptoms that resolved more quickly than those with higher certainty, thus not requiring a return ED visit.
Our finding of the lack of association between the total U-Scale score and recurrent 30-day ED visits, yet a significant association with two of the U-Scale subscale scores and subsequent utilization suggests that uncertainty is not a unitary phenomenon and that only some aspects of uncertainty are predictive of future utilization. The obtained effect sizes for the Treatment Quality and Decision to Seek Care subscales (0.41 and 0.46, respectively) are medium effects that will inform future scale development work determining cutoff values that might be used in clinical screening or intervention.
Limitations
This study has several limitations. We enrolled a convenience sample from a single ED located within a large urban academic hospital to recruit respondents who sought care in the ED and did not get hospitalized. More than half of the sample was black, female, and earned less than $49, 999 annually, so findings may not be generalizable to other populations, although our sampling frame likely resulted in a relatively representative sample from our population.
We assessed utilization by patient self-report. Data available to our team did not include administrative claims related to ED use at health systems outside the index institution where this study was performed. Given that acute care utilization patterns are often independent of health system boundaries,35 especially in the geographic region where this study was performed, which has multiple health systems in one urban region, single-site administrative data would like underestimate repeat visits. As such, the team decided that patient self-report was preferable to the alternative of chart review, in which we would miss visits occurring outside the health system. With patient self-report, we acknowledge that there is potential for bias due to both recall bias and patient preferences to either under- or overreport their utilization. Recall bias in self-report of health care utilization may be influenced by factors such as patient cognitive abilities, recall time frame, type and frequency of utilization, questionable design, and mode of data collection.36 Prior studies of concordance between self-report and insurance claims specifically for ED visits suggest good to very good concordance recall within a year,37, 38 and the impact of memory decay on recall in our study may be mitigated by the relatively short recall time (30 days).
Our sample size was relatively small, with a ratio of cases to predictors of approximately 15:1. Despite this, we did find two significant effects, both of medium magnitude. It is possible that other effects were missed owing to low power and we plan to address this in follow-up work. Our sample size was reduced by the fact that close to one-quarter of the enrolled sample were unable to be contacted for follow-up. Although our analyses indicate that those lost to follow-up were not demographically different from those we were able to follow-up in any of the characteristics we identified, it remains possible that this group could have different in factors not captured by these variables. We are working with the regional healthshare data exchange for future work to facilitate more accurate collection of subsequent utilization data.
In addition, these data were collected as part of initial scale refinement and validation work, and the factor analysis resulted in subscales/subdomains of patient uncertainty measured with as few as two items. Our subsequent work is focused on scale refinement including the potential addition of items within subdomains. This will be followed by further exploratory and confirmatory factor analyses as well as predictive modeling to establish validity of the subscales. Although some subscales consisted of only a few items they demonstrated reasonable internal consistency and showed strong associations with the outcome. Despite these limitations, to our knowledge, this is the first study to assess patient uncertainty as a predictor of return ED visits. Findings suggest that uncertainty may be a predictor of ED returns and that this association warrants further study with a larger sample.
Conclusion
In summary, our findings suggest that patient uncertainty during an ED visit related to treatment quality and the decision to seek care may be important in predicting future ED utilization. Importantly, unlike many patient factors that have been identified as associated with frequent ED visits, such as age and insurance status, these patient domains of uncertainty are potentially modifiable. Through application of interventions designed to specifically address these unmet patient needs, providers and health systems may successfully prevent recurrent care cycles. As such, the next steps include continued refinement of the scale by testing in different geographic locations and using a larger sample. Our developmental work with the U-Scale is ongoing, and the finding of predictive utility of two elements of uncertainty as measured by the U-Scale provides evidence that the concept of uncertainty is viable and that patient uncertainty contributes to the decision to seek care.
