1.1 High-Dimensional Propensity Score

Residual confounding is pervasive in analyses of health administrative databases, where unmeasured or imperfectly measured confounders can bias effect estimates. The high-dimensional propensity score (hdPS) method is widely used in pharmacoepidemiology and health outcomes research to leverage large administrative databases capturing diverse healthcare interactions (e.g., diagnostic codes, prescriptions, procedures) [1]. Unlike traditional propensity score approaches that rely exclusively on pre-specified covariates [2], hdPS systematically identifies and includes proxy confounders from high-dimensional data [3]. Known confounders (e.g., those in causal diagrams) are always retained a priori, ensuring that critical variables are never subjected to data-driven selection. By integrating prior knowledge with automated proxy selection, hdPS improves control of both measured and unmeasured confounding, capitalizing on the richness of large healthcare datasets while maintaining interpretability.

1.2 Extensions and Gaps

The original hdPS algorithm uses Bross's formula to rank potential proxies according to their marginal associations with treatment and outcome [4]. However, high-dimensional and correlated administrative data can lead to overfitting and multicollinearity. To address these issues, multi-variate and machine learning alternatives have been proposed [5]. Examples include regularized regression (e.g., LASSO, elastic net) [6, 7], tree-based methods [7, 8] and ensemble methods (e.g., super learner) that incorporate flexible modeling strategies [9, 10].

Some studies have used targeted maximum likelihood estimation (TMLE), which combines flexible machine learning for both treatment and outcome models. However, logistic regression is still commonly chosen as the sole learner within TMLE applications for computational reasons [11, 12]. This means the full potential of TMLE with a diverse super learner remains a relatively unexplored area in the hdPS context. While double-robust methods (e.g., TMLE) have outperformed singly robust approaches in low-dimensional settings with investigator-specified covariates [13, 14], it remains unclear whether these findings extend to high-dimensional contexts, especially when the number of potential proxies is large (possibly including noise variables) and model specification is complex. Moreover, the performance of TMLE can depend on the choice of candidate learners in the super learner [15], and additional sample-splitting (e.g., double cross-fitting) may be required to ensure robust estimation [16, 17]. However, the computational burden and practical positivity challenges of double cross-fitting approaches can be substantial, particularly in large datasets. This may explain the lack of implementation of these newly proposed double cross-fitting methods in the hdPS literature.

1.3 Aim

Despite numerous proposed enhancements to hdPS-related methods, critical gaps remain in understanding their comparative performance when integrated with machine learning and doubly robust estimators. Specifically, evaluating how these approaches perform under varying data complexities, exposure and outcome prevalence, and different candidate learner configurations is essential for guiding practitioners in designing robust analytical protocols. This study aims to address these gaps by systematically assessing these methods under realistic conditions using plasmode simulations. To further contextualize our findings, we will apply these methods to real-world data from the National Health and Nutrition Examination Survey (NHANES) [18] (2013–2018), providing empirical insights into their practical applicability.

2.1 Motivating Real-Wold Example

Obesity is a well-established risk factor for type 2 diabetes, with excess body fat contributing to insulin resistance and impaired blood sugar regulation [19]. To investigate this relationship, we will use data from NHANES. The dataset includes 7585 individuals [1], of whom 48.8% were categorized as obese (binary exposure) and 23.7% had diabetes (binary outcome).

The dataset includes 14 demographic, behavioral, and health history-related variables (mostly categorical) and 11 laboratory-measured variables (mostly continuous) as measured confounders. However, residual confounding may still arise from unmeasured comorbidities or other health conditions not captured in the survey. To address this limitation, we leverage the NHANES Prescription Medications component, which provides detailed data on prescription medications taken by participants prior to the survey. This information can serve as a crude proxy for comorbidity burden, helping to account for unmeasured health conditions [20-22].

In this context, methods under the hdPS framework have the potential to reduce residual confounding by systematically incorporating proxy information derived from prescription medication data. By integrating these proxies into the analysis, we aim to improve our estimates and better isolate the causal effect of obesity on diabetes.

2.2 Methods Under Consideration

We evaluated the association estimates using the hdPS method [3] and its alternatives, as summarized in Table 1 (see Supporting Information: Appendix §A for further details). Several of these approaches, such as LASSO [6, 7], super learner [9], and TMLE [11, 12], have been previously proposed and implemented in the literature. Standard methods, including hdPS, LASSO, and a hybrid hdPS-LASSO approach, were implemented similarly to their original implementations in the literature [6, 7], with the exception that we used inverse probability weighting (IPW) instead of including propensity scores as covariates in the outcome regression.

For the non-TMLE methods, we evaluated performance using different adjustment sets in the weighted outcome model to understand the impact of various levels of adjustment on bias and coverage. Five distinct adjustment strategies were considered: no adjustment, imbalanced measured confounders, all measured confounders, imbalanced measured confounders with proxies, and all measured confounders with proxies. The “no adjustment” scenario served as a baseline, with no confounders or proxies included, allowing us to observe the effects of residual confounding. For these analyses, we explored both stabilized and unstabilized weights to assess their potential impact on performance metrics.

However, implementations involving super learner and TMLE varied significantly across studies (because candidate learners were different), making direct comparisons challenging. For instance, some applications of super learner utilized 23 candidate learners [9], while others used only logistic regression for propensity score estimation [11, 12]. Additionally, methods such as double cross-fitted TMLE have been implemented in low-dimensional settings [16, 17] but remain largely unexplored in the hdPS context.

2.3 Plasmode Simulation

To rigorously evaluate the performance of the methods under consideration, we employed a plasmode simulation framework, which is particularly well-suited for replicating real-world data structures and complexities [23]. This approach was inspired by the analytic dataset derived from NHANES and involved resampling observed covariates and exposure information (i.e., obesity) without modification. By preserving key aspects of an actual epidemiological study, this simulation framework offers a significant advantage over traditional Monte Carlo simulations, which often rely on hypothetical assumptions.

3.1 Motivating Example

The results are shown in Figure 1. Note that, other than the methods that ignored the proxies completely (with a u at the end), the rest of the methods provided very similar point estimates.

3.2 Plasmode Simulation Results

Figures 2 and 3 illustrate the bias and 95% coverage estimates across frequent, rare exposure, and rare outcome scenarios. The results are summarized below.

4.1 Motivating Example

Methods that incorporated high-dimensional proxies, such as standard hdPS, TMLE, and SL methods, yielded similar point estimates for the association between obesity (exposure) and diabetes (outcome), regardless of the candidate learner library used. Methods without proxies showed notable deviations in their estimates. While this motivating example highlighted general trends in estimator performance, it was limited to a single dataset and lacked the capacity to systematically assess bias, variance, and coverage. In contrast, the plasmode simulation provided a comprehensive evaluation of these statistical properties across a range of realistic scenarios, offering deeper insights into estimator behavior in high-dimensional contexts.

4.2 Simulation Results

Our plasmode simulation revealed distinct behaviors across general groups of methods. Methods without proxies (PS.u, SL.u, TMLE.u) performed the worst across all scenarios, showing consistently high bias and severely under-calibrated coverage. These findings emphasize the critical role of proxies in confounding adjustment, especially in the presence of unmeasured confounding. In contrast, standard methods incorporating high-dimensional proxies (e.g., PS.ks, hdPS, LASSO, hdPS.LASSO) demonstrated substantial improvements in bias and coverage, achieving near-nominal coverage in most scenarios.

SL methods (hdPS.SL, LASSO.SL, hdPS.LASSO.SL, SL.ks) showed flexibility and reasonable bias performance but experienced significant coverage deterioration when non-Donsker learners were included in the library. TMLE methods (hdPS.TMLE, LASSO.TMLE, hdPS.LASSO.TMLE, TMLE.ks) exhibited robust performance with low bias and reliable coverage for smooth learners. However, as with SL methods, coverage markedly declined when complex learners, such as XGBoost, which do not belong to the Donsker class, were included. Among TMLE methods, DC TMLE consistently achieved the lowest empirical SE and well-calibrated coverage in most scenarios, yet underperformed in high-dimensional settings when non-Donsker learners were included.

The SL framework, particularly its integration within the TMLE framework, is widely recognized for its flexibility and ability to incorporate diverse learners [5]. However, our findings highlight key nuances in its application to high-dimensional proxy adjustment. While previous SL tutorials emphasize the benefits of library diversity [29, 30], our results suggest that increased complexity is not always beneficial. Specifically, simpler libraries, such as the 1-learner and 3-learner configurations, consistently outperformed larger libraries in terms of bias and coverage, achieving greater stability across all scenarios. The 1-learner library avoided overfitting and adhered to theoretical assumptions, while the 3-learner library (logistic regression, LASSO, and MARS) performed similarly but occasionally exhibited slightly higher bias and under-calibrated coverage.

Conversely, the 4-learner library, which included XGBoost (a non-Donsker learner), introduced significant variability, resulting in increased bias and poor coverage, particularly in rare exposure scenarios. These findings align with guidelines advocating for tailoring SL libraries to effective sample size [15] and underscore the importance of including simpler algorithms, such as regression splines and generalized linear models, in SL libraries [13]. While DC TMLE has demonstrated advantages in low-dimensional settings [17], its performance in high-dimensional settings with complex learners was unsatisfactory. This suggests that larger libraries, though flexible, may introduce instability, especially when incorporating learners that violate Donsker conditions.

4.3 Model Complexity Versus High-Dimensionality

The strong performance of standard methods incorporating high-dimensional proxies (e.g., PS.ks, hdPS, LASSO, hdPS.LASSO) across all scenarios highlights the value of simple yet effective modeling strategies in high-dimensional settings. These methods benefit from directly leveraging proxies for confounding adjustment, which—when selected through structured algorithms such as hdPS or LASSO—can capture key confounding signals without adding unnecessary complexity. This simplicity mitigates overfitting and instability [31], leading to bias reduction and near-nominal confidence interval coverage.

While ensemble learners such as SL offer modeling flexibility, they may also introduce greater variance—particularly when combined with unstable or non-smooth base learners. Similarly, the iterative targeting step of TMLE, though designed to improve efficiency, may exacerbate small-sample variability in the presence of high-dimensional covariate spaces or complex learners [32]. For TMLE methods, thoughtful selection of library components (favoring smooth learners) may help mitigate these issues [13, 15].

These observations align with semiparametric theory, which warns that greater model flexibility often comes at the cost of slower convergence rates. Nonparametric and machine learning estimators typically converge more slowly than parametric models, and this can result in inflated estimation error and poor finite-sample performance—particularly when both treatment and outcome models must be estimated non-parametrically [33]. This tradeoff is especially critical in doubly robust estimators such as TMLE, which require adequate convergence of both nuisance functions.

These challenges are exacerbated in high-dimensional settings due to the curse of dimensionality and increased estimation instability. As our results demonstrate, higher modeling complexity does not necessarily lead to improved statistical properties. In fact, it can degrade performance when flexible learners are combined with iterative procedures such as the TMLE targeting step. These findings reinforce theoretical guidance: added flexibility must be carefully balanced against potential variance inflation and instability—especially in complex, high-dimensional causal inference problems.

4.4 Strengths, Limitations, and Future Directions

This study provides a systematic evaluation of high-dimensional proxy adjustment methods combined with machine learning and doubly robust estimators. By leveraging realistic plasmode simulations, we examined estimator performance across varying exposure and outcome prevalence scenarios. Unlike prior studies that focused on low dimensions or investigator-specified covariates, our work extends the literature by assessing diverse adjustment strategies and learner configurations. Importantly, we highlight the challenges posed by non-Donsker learners within the hdPS framework and their impact on bias, coverage, and stability.

However, this study has several limitations. We did not assess overlap between treatment groups or conduct diagnostics, such as covariate balance checks, across all simulations, as recommended by transparency guidelines [34-36]. Additionally, the findings are shaped by the data structure of the plasmode simulation, which may not fully capture complexities such as time-varying confounding or extreme sparsity. While our study observed under-coverage with DC TMLE in high-dimensional settings, prior work suggests that single cross-fit TMLE, cross-validated TMLE (CVTMLE) and its variants (e.g., CVTMLE[Q]) could mitigate these issues by stabilizing variance and improving confidence interval coverage [37].

Some previous studies have included collaborative targeted maximum likelihood estimation (C-TMLE) [38] as a comparable method to TMLE, with a specific emphasis on its iterative variable selection procedure aimed at reducing confounding bias [10, 39]. We chose not to include this method in our current work due to practical and methodological considerations specific to high-dimensional proxy adjustment settings.

While C-TMLE offers theoretical advantages in iterative covariate selection and bias reduction, its computational complexity poses significant challenges in high-dimensional datasets, where the numbers of covariates and proxies are large. Although scalable versions of C-TMLE exist, these rely on preordering strategies, such as logistic preordering or partial correlation approaches, which can introduce arbitrary assumptions about the importance or influence of covariates [40]. This ordering could be particularly problematic in high-dimensional proxy settings, where the relationships among variables can be complex, and any imposed ordering may fail to capture the true confounding structure. We also did not consider deep learning algorithms, such as neural networks [41, 42], due to their complexity and potential instability in high-dimensional settings.

Screener-based approaches could also help mitigate instability caused by non-Donsker learners by reducing dimensionality prior to applying SL [15]. Extending these methods to more complex contexts, such as time-varying exposures or longitudinal data [43], represents an important direction for future research. Recent work has highlighted unique challenges in these settings, including dynamic confounding and data sparsity [13]. Addressing these complexities will enhance the applicability of these methods to a wider range of epidemiological studies.

5.1 Plain Language Summary

Observational studies often face challenges in estimating causal relationships due to unmeasured confounding—factors that influence both treatment and outcome but are not directly observed. hdPS methods help address this issue by using proxy variables that indirectly capture unmeasured confounders. This study evaluates how well different approaches—ranging from traditional hdPS methods to machine learning-based estimators—perform under various data conditions. Using realistic simulations, we find that incorporating high-dimensional proxies significantly improves bias reduction and coverage. While methods such as TMLE and super learner offer advantages, their performance depends on the choice of models. More complex configurations, particularly those including advanced learners such as XGBoost, may introduce instability, leading to reduced accuracy. Our findings suggest that simpler models, including traditional hdPS with proxies, often provide robust and reliable results. We also applied these methods in a real-world data from the NHANES (2013–2018). These results offer valuable guidance for researchers applying statistical methods to reduce confounding bias in epidemiological studies.