Improving Racial Equity in the Veterans Health Administration Care Assessment Needs Risk Score
Abstract
Research Objective
The VA computes the Care Assessment Needs (CAN) score weekly for over 5 million Veterans to predict risk of one-year mortality and to improve resource allocation to high-risk Veterans. Motivated by evidence of unfair predictive algorithms in other settings, our objective was to examined the CAN score for racial unfairness.
Study Design
We constructed a cross-sectional cohort of Veterans who were alive and had at least one outpatient primary care encounter during 2016, based on a VA national repository of administrative claims and electronic health data containing inpatient, outpatient, laboratory, procedure, and pharmacy encounters. We used the last score of the CAN 2.5 model (current CAN version) in 2016 for all analyses. First, we descriptively compared distributions of the last CAN scores in 2016 for self-identified White and Black Veterans. Second, we assessed CAN fairness by calculating the false-negative rate (FNR) as our primary fairness metric, defining a “positive” prediction at or above the 80th percentile for Black and White Veterans. Deaths were confirmed using 2017 mortality data. Third, to investigate contributors to unfairness, we compared pooled mortality within strata of Black and White Veterans based on exact matches of the most influential variables in the CAN model: age and Elixhauser comorbidities. To account for class imbalance (lower representation of Black Veterans) we re-assessed fairness after re-training the CAN model by upweighting the Black cohort.
Population Studied
Our population consisted of 791,438 (18.3%) Blacks and 540,877 (81.7%) Whites.
Principal Findings
Black Veterans were younger (median age 59 vs. 67) and more likely to suffer from PTSD (30.9% vs. 22.4%) and be unmarried (58.8% vs. 42.9%). CAN scores were lower for Blacks than Whites (mean [SD] 41.8 [28.2] vs 52.2 [28.1]) and appeared more unfair for Blacks than Whites (FNR 35.3% vs. 26.5%, meaning CAN under-predicted death for Blacks versus Whites). When matching on comorbidities, the pooled mortality rate was lower for Blacks (2.1% vs. 3.6%), largely because younger Blacks had similar comorbidities to older White Veterans. This discrepancy was mitigated after additionally matching on age (pooled mortality 2.9% vs. 3.0%). Accounting for class imbalance marginally reduced unfairness for Blacks vs. Whites (FNR 34.1% vs. 25.4%).
Conclusions
The CAN score, a widely-used VA risk model, underestimates mortality risk for Black relative to White Veterans. Differences in the age distributions strongly suggest statistical unfairness driven by confounded social factors. Addressing class imbalance only marginally improves fairness.
Implications for Policy or Practice
This is the first study to show systematic racial unfairness in a VA algorithm due to a relatively young and sick Black population, a mechanism of unfairness that could apply to other care management algorithms. Mitigating algorithmic unfairness may require data on social determinants of health and should be a priority to improve VA healthcare equity.
Primary Funding Source
Department of Veterans Affairs.
