2.1 Motivating Example: Tuberculosis Cohort in British Columbia

We utilized a retrospective population-based cohort of immigrants in British Columbia, Canada, between 1985 and 2019, who were tested for TB infection as coded in the provincial TB registry. The cohort was developed using linked immigration, TB Registry, and health administrative databases (Appendix-A) [4, 16]. The final analytical sample consisted of 49 197 individuals tested for TB infection, totalling 901 734 person-years of observation. Among these participants, 26 163 (53.2%) tested positive for TB infection, and 2114 (4.3%) developed CVD during the follow-up period. Fourteen measured confounders were considered for confounding adjustment (Appendix-A) [4].

2.2 Methods Compared

We focused on contrasting confounder adjustments using hdDRS, and also explored results from hdPS and multiple regression adjustments (Table 1). For the multiple regression analysis in a traditional way, the Cox-PH regression was fitted as the outcome model, adjusting for measured confounders. Compared to the traditional propensity score (PS) and DRS approaches that rely solely on investigator-specified measured confounders, hdPS and hdDRS methods systematically leverage high-dimensional healthcare databases to empirically identify and select additional proxy covariates that serve as surrogates for important, often unmeasured confounders.

There are seven steps in hdPS and hdDRS analyses, such as identifying the data sources for proxies, dimension reduction, defining recurrence covariates (‘empirical covariates’) from proxies, prioritizing empirical covariates, selecting top empirical covariates, estimating the PS or DRS, and outcome modelling [5-7, 13]. The steps are described in Table 1. Steps 1–5 are identical for both the hdPS and hdDRS methods; however, step 6 (score estimation) differs between them. Step 6 involved estimating the PS (the conditional probability of receiving the exposure given measured confounders) in the hdPS analysis, and the DRS (the conditional probability or rate of developing the outcome given measured confounders) in the hdDRS analysis. To estimate the PS or DRS, the measured confounders and the 200 empirical covariates identified using Cox-LASSO were used (Appendix-B). While logistic regression was used to estimate the PS, we calculated the DRS in eight different approaches: Full-Logistic, Full-Survival, Full-Hazard, Full-Rate, Unexposed-Logistic, Unexposed-Survival, Unexposed-Hazard, and Unexposed-Rate (Table 1) [14, 15]. Step 7 involved the outcome modelling, where the Cox-PH model was fitted, adjusting for the measured confounders and the deciles of the PS or DRS (Appendix-C). White's robust sandwich estimator was used to calculate the model-based standard error for both hdPS and hdDRS methods [17]. To demonstrate how to apply the compared methods in a given scenario, reproducible R codes on a simulated dataset are provided in the GitHub folder: https://github.com/belalanik/hdDRScausal.

2.3 Simulation Setup

3.1 Strong Unmeasured Confounding Present

All methods produced a biased effect estimate for a strong unmeasured confounding scenario. The bias was −0.11 for all methods when only investigator-specified confounders were used in all analyses (Appendix-Figure S2). When empirical covariates were added to the investigator-specified confounders, the bias was significantly reduced for all hdDRS methods (Figure 1, Table 2). In contrast to all hdDRS methods, the hdDRS with rate-based DRS calculation on the unexposed cohort resulted in the lowest bias of −0.045, while the hdDRS with logistic-based DRS calculation on the full cohort had a bias of −0.058. The hdPS resulted in a bias of −0.047 in the same setting.

The model-based and empirical SE were comparable for all methods (range: 0.041–0.043). As expected, all methods had poor coverage probabilities (0.22 for traditional, 0.68–0.78 for hdDRS methods, and 0.78 for hdPS). However, the bias-eliminated coverage probability was nominal for all methods (Table 2). Although all hdDRS methods had comparable SE and coverage probabilities, fitting the DRS models only on the unexposed individuals rather than the full cohort resulted in more bias reduction (range from −0.045 to −0.049 versus −0.051 to −0.058). All methods had a very small Monte Carlo error, with an error of 0.001–0.003 for bias, 0.001–0.002 for empirical SE, 0.000 for model-based SE, 0.013–0.034 for coverage, and 0.006–0.014 for bias-eliminated coverage.

3.2 Weak Unmeasured Confounding Present

For a weak unmeasured confounding scenario, the bias was small for all methods (Figure 2, Table 2). The bias was −0.020 for the traditional method, while it was −0.006 for the hdPS and ranged from −0.014 to −0.001 for the hdDRS methods. In contrast, the hdDRS methods with full-cohort DRS score calculation resulted in a bias between −0.008 and −0.014, while the hdDRS methods with unexposed-only DRS score calculation resulted in a bias between −0.001 and −0.005. The bias was −0.001 for the hdDRS with rate-based DRS calculation on the unexposed cohort.

The model-based and empirical SE were comparable for all methods (0.041–0.043). The traditional method had a coverage probability of 0.91, while it was 0.94 for the hdPS and ranged from 0.93 to 0.95 for the hdDRS methods. The bias-eliminated coverage probability was nominal for all methods (Table 2). Again, the Monte Carlo error was small for all methods, such as 0.001–0.004 for bias, 0.001–0.003 for empirical SE, 0.000 for model-based SE, 0.007–0.020 for coverage, and 0.007–0.018 for bias-eliminated coverage.

3.3 Results for Sensitivity Analyses

The results of the sensitivity analyses were not materially different compared to the primary analysis (Appendix-F). Similar to the primary analysis with a strong unmeasured confounding, the hdDRS with rate-based DRS calculation on the unexposed cohort consistently produced the lowest bias for varying the event rate and exposure prevalence (Appendix-Table S2), with 500 empirical covariates or ranking the empirical covariates using the Bross formula or random survival forest (Appendix-Table S3), for a null exposure effect (Appendix-Table S4), misspecified DRS and PS models (Appendix-Table S5), LASSO for estimating scores (Appendix-Table S6), risk difference as the target parameter (Appendix-Table S7), and when data were generated for 5000 individuals (Appendix-Table S8), 2000 individuals (Appendix-Table S9), 1000 individuals (Appendix-Table S10), and age and sex as the only confounders (Appendix-Table S11). The hdDRS with rate-based DRS calculation on the unexposed cohort and the hdPS also produce approximately unbiased estimates with a weak unmeasured confounding.

3.4 Motivating Example Results

In exploring the relationship between TB infection and CVD, the traditional model had an adjusted HR of 1.08 (95% CI: 0.99–1.18). The HR was 1.08 (95% CI: 0.99–1.18) for the hdPS and 1.09 (95% CI: 1.00–1.19) for the hdDRS with rate-based DRS calculation on the unexposed cohort. The other hdDRS methods resulted in an HR ranging from 1.08 to 1.11 (Figure 3).

4.1 Summary of the Findings

In a scenario with strong unmeasured confounding, the traditional method that relies on adjusting the model for only the measured confounders always produced a biased effect estimate. Adding empirically identified proxies to the investigator-specified measured confounders and fitting the hdDRS resulted in a significant bias reduction. The hdDRS with rate-based DRS calculation on the unexposed cohort consistently exhibited the least bias. The model-based and empirical SE were comparable for all methods. The bias-eliminated coverage probabilities also approached nominal levels across all methods. In a setting with weak unmeasured confounding, all methods performed well, demonstrating minimal bias, especially for the hdDRS with DRS calculation on the unexposed cohort. The hdPS also had results comparable to those we obtained from the hdDRS with DRS calculation on the unexposed cohort. Sensitivity analyses for varying the exposure prevalence and event rate, different hdDRS parameters, null exposure effect, model misspecifications, LASSO for estimating the scores, risk difference as the effect measure, smaller sample sizes, and specifying the true data-generating model with only age and sex revealed consistent trends; the hdDRS with rate-based DRS calculation on the unexposed cohort generally outperformed the other methods in reducing bias due to residual confounding. We applied all the methods in exploring the relationship between TB infection and CVD among people who immigrated to British Columbia and were tested for TB infection between 1985 and 2019. The method with only investigator-specified measured confounders showed an 8% higher risk of CVD among people who tested positive for TB infection than those who tested negative. Adding the empirically identified proxies with the investigator-specified measured confounders and fitting the hdPS and hdDRS methods showed an 8%–11% higher CVD risk associated with TB infection.

4.2 HdDRS to Deal With Residual Confounding Bias

The hdDRS methods with DRS calculation on the unexposed cohort had a comparatively smaller bias than the hdDRS methods with DRS calculation on the full cohort. Since DRS is used to balance baseline outcome risks, fitting the DRS models only among the unexposed and predicting the score on the full cohort could offer a stable estimate of the baseline risk of the outcome among the exposed and unexposed. This approach could also better capture the baseline risk of the outcome since control for confounding might otherwise be different in the exposed group [14]. Moreover, disentangling the risk of the outcome associated with proxies and exposure could be challenging when we include the exposure variable in the DRS calculation. Notably, the hdDRS-Unexposed-Logistic ignores the follow-up time in estimating the DRS; the hdDRS-Unexposed-Survival attempts to balance the survival probability of the outcome during the follow-ups; the hdDRS-Unexposed-Hazard attempts to balance the hazard during the follow-ups, while the hdDRS-Unexposed-Rate only attempts to balance the baseline outcome risks. The better balance of the baseline risk might be the reason for the consistently lower bias from the hdDRS with rate-based DRS calculation on the unexposed cohort than other hdDRS methods with DRS calculation on the unexposed cohort.

The hdPS is a popularly known method compared to the hdDRS in dealing with residual confounding bias in comparative effectiveness studies. Our study also found that the hdPS had comparable performance to the hdDRS with rate-based DRS calculation on the unexposed cohort in terms of reducing bias. However, the instability of an overfitted exposure model with a rare exposure in hdPS could fail to reduce significant bias [25]. The hdDRS methods with DRS calculation on the unexposed cohort could perform well in the same setting when the outcome of interest is not rare. Notably, both hdPS and hdDRS could perform poorly and overestimate the exposure effect when the PS and DRS models are overfitted, particularly with a rare exposure and a rare outcome, respectively [26, 27]. These methods also assume that the empirical covariates are potentially correlated with the variables that are unmeasured and these empirical covariates can be used as overall proxy measures to minimize bias due to residual confounding [5, 8, 28]. However, both approaches could be as good as the traditional method, even if there is strong or weak confounding or some confounders are misclassified. As we observed in our simulations, hdPS and hdDRS with rate-based DRS calculation on the unexposed cohort always resulted in approximately unbiased estimates under a weak unmeasured confounding scenario, while there were still some biases from the traditional method. In practice, rare outcomes are more common than rare exposures in comparative effectiveness studies with observational data. We recommend applying both hdPS and hdDRS with rate-based DRS calculation on the unexposed cohort. The analyst could come to a robust conclusion if both approaches result in a similar effect estimate.

4.3 HdDRS in Literature

Many previous studies explored the performance of PS and DRS methods developed with only investigator-specified measured confounders [15, 29-32]. The performance of the PS and DRS compared to that of a traditional method is not always superior, particularly when both exposure and outcome are non-rare [15, 29, 30]. Most of the hdPS work applies to continuous or binary outcomes, while only a few studies have been conducted with a survival outcome [9, 11, 33, 34]. However, the literature on hdDRS in dealing with residual confounding is limited. We found only two empirical studies on hdDRS, and both considered binary outcomes [7, 13]. One study showed that hdDRS using historical data improved confounding adjustment [7], while the other study concluded that hdDRS had similar or better confounding adjustment compared to the traditional method but worked slightly less well than hdPS [13]. However, both studies were empirical in nature without any simulations. The hdDRS was also applied in many applications, such as dealing with unmeasured/residual confounding in TB [4], cataract [35], fracture [36], CVD [37], diabetes [38], and multiple sclerosis research [39]. When exposure is time-dependent, or the outcome is being measured over time, implementing hdPS is challenging. In a recent study, we employed hdPS within the nested case–control design to simultaneously deal with time-dependent exposure and residual confounding [40]. The hdDRS methods can also be used in the same setting. With a longitudinally measured outcome, hdPS might not be implemented since each individual has multiple rows of data. However, the hdDRS methods with DRS calculation on the unexposed cohort could easily be applied in the same setting, which was also done in a recent work minimizing residual confounding bias with an outcome occurring over time [41].

4.4 Contextualize the Findings in TB Literature

Our results consistently showed that TB infection is associated with a modestly increased risk of CVD across methods. While the traditional model showed an 8% higher CVD risk (HR: 1.08, 95% CI: 0.99–1.18), the hdPS and hdDRS methods produced hazard ratios ranging from 1.08 to 1.11. The small variation across methods suggests relatively weak unmeasured confounding by smoking status, highlighting the robustness of the observed association. However, if the empirically identified proxies were poor surrogates for smoking, residual confounding may persist. We observed an 11% higher risk of CVD associated with TB infection in a previous study [4], where we conducted a sensitivity analysis by adjusting our model with investigator-specified confounders as well as tobacco use as a proxy for smoking. All these findings underscore the potential effect of TB infection on long-term CVD risk, while also demonstrating the importance of employing robust statistical methods to address residual confounding bias inherent in observational studies.

4.5 Limitations and Future Directions

Our study has some limitations. First, the hdPS and hdDRS methods might minimize residual confounding bias inherent in using health administrative data, while some residual confounding may remain. As we observed in our simulations, all methods produced a biased estimate with strong unmeasured confounding, while the bias was approximately zero in hdPS and hdDRS analyses with weak unmeasured confounding. Second, we considered the log-HR as the target parameter, which is a non-collapsible effect measure. Our analysis without unmeasured confounding revealed that all these methods resulted in a similar log-HR with a similar SE. The log-HR and associated SE were also similar for all methods when only the measured confounders were used in fitting the model. Moreover, our sensitivity analysis with the risk difference as a collapsible effect measure had similar trends to the results that we observed with the log-HR as the target parameter. This study estimated conditional (covariate-adjusted) effects of exposure rather than marginal (population-average) effects. Conditional estimates reflect the causal effect within subgroups defined by specific covariate patterns or strata and may not directly generalize to the entire target population. To obtain marginal effects, alternative methods such as inverse probability weighting or G-computation would be required. Therefore, caution is warranted when interpreting the current estimates as representative of the causal effect in the broader population. Third, we adjusted the outcome model for the deciles of the DRS, which is also commonly done in previous studies [7, 13, 29]. The stratification approach for hdPS and hdDRS, as well as hdPS with IPW, had similar effect estimates. Future studies could compare the methods with different modeling choices such as matching and weighting. Future studies could also combine PS and DRS in the same model and explore the performance with the separate hdPS and hdDRS methods [42]. Fourth, the inclusion of many covariates in hdPS may lead to a violation of the positivity assumption. The violation of the positivity assumption can result in a biased effect estimate as well as an inflated variance estimate [26, 43]. Trimming, overlap weighting, extrapolation beyond the overlap region, matching with a caliper, stratification, and estimating PS using machine learning dimension reduction techniques to reduce the dimension of covariates are often recommended to address the violation of the positivity assumption [44, 45]. Fifth, we considered the conventional methods for estimating PS or DRS. Future studies could estimate PS or DRS using machine learning methods. Since such approaches often result in poor coverage, recent literature has recommended using double robust methods, such as Targeted Maximum Likelihood Estimation [46-48]. However, we observed approximately nominal bias-eliminated coverage when PS or DRS was estimated using LASSO. Furthermore, the literature on machine learning methods for survival outcomes is limited [49], and the implementation details of double robust methods are somewhat unclear [50]. Sixth, we did not consider competing events in our analysis, while future studies could explore the performance of various versions of hdPS and hdDRS with a competing event. Given that hdDRS and hdPS are often specific to the data and covariates available, our results might not be generalizable. We recommend exploring the performance of hdPS and hdDRS in different research contexts by varying key hdPS/hdDRS parameters.

5.1 Plain Language Summary

This study aimed to explore whether tuberculosis infection causes heart disease. People with tuberculosis have different characteristics than those without it. If we do not know some of these important factors, our exploration of the relationship can be biased. For example, we did not have smoking information in our data. The traditional approach only considers known, measurable factors, leading to a biased relationship. There are many healthcare variables available for patients. For example, diagnostic and procedure codes and drug dispensation can be useful. These variables can provide valuable information about smoking. We used these variables with advanced methods. This was to make up for the lack of data on smoking. A technique called the high-dimensional disease risk score was very effective. It worked best when the score calculation used data from uninfected people. Our method worked well even though some other important factors were unobserved. After fixing the bias, we found an 8% to 11% higher risk of heart disease from tuberculosis infection.