Patient study cohort

The training cohort for this study was obtained from the ACS-NSQIP pancreatectomy-targeted database from 1 January 2014 to 31 December 2017, which registers up to 274 perioperative indicators. ACS-NSQIP is a prospectively maintained multicenter database whose data collection is standardized, validated, and reliable.¹⁹ More information about the database, data collection protocols, and definitions is available online (https://www.facs.org/quality-programs/acsnsqip/participant-use). The training cohort included PDAC patients who underwent elective PD using CPT codes: 48150, 48,152, 48,153 and 48,154. Figure S1 presents the data distribution features in the training cohort. The validation cohort was derived from the data of PDAC patients who underwent PD in the pancreatic center of Chinese NCC hospital from 1 January 2014 to 31 December 2019. Cases without PD, pathological non-PDAC, or cases with missing data on variables were excluded from the analysis. Furthermore, indicators related to the diagnosis of DGE, such as postoperative gastric tube reset, which may amplify the predictive power of the model, were also excluded. Additionally, indicators without potential clinical significance based on clinical experience and previous reports, such as nursing methods, or those with missing data ≥10%, were also excluded. Figure 1 presents a flowchart of the data screening and analysis. The design, validation and reporting of this clinical prediction model followed the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) guidelines of the multivariable prediction model.²⁰ The study exclusively utilized non-identifiable pre-existing data, and as a result, it was determined to be exempt from the requirement for informed consent procedures. This exemption was granted in accordance with the Declaration of Helsinki and the Ethical Guidelines for Clinical Studies (No. P-2021-1230). This study was approved by the Ethics Committee of Chinese NCC (No. 17-168/1424). This retrospective study was registered with ResearchRegistry.com (Unique Identification Number: researchregistry8530). Data has been reported in line with STROCSS 2021 criteria.²¹

Definitions and indicators

DGE is defined according to the 2007 definition by the International Study Group of Pancreatic Surgery (ISGPS).⁴ POPF is defined based on the 2016 ISGPS definition.²² According to these definitions, both DGE and POPF are classified into three grades: A, B, and C. Grade A is typically not clinically significant. Therefore, in this study, DGE and POPF refer to clinically relevant grades B and C. Since only a small number of patients were classified as grade C (Table 1; Figure S2a,b), further subclassification was not performed during the modeling process. In light of the lack of broad consensus on the definitions of primary and secondary DGE, we did not stratify DGE in this manner. The definitions of POPF and DGE in ACS-NSQIP are consistent with ISGPF (https://www.facs.org/media/vcdilnwc/pt_nsqip_puf_userguide_2017.pdf). The study cohort consisted of a total of 46 perioperative indicators, which included 20 preoperative variables, six intraoperative variables, and 20 postoperative variables. The extent of lymphadenectomy for the study cohort followed the 2014 ISGPS guideline.²³ The specific indicators are shown in Table 1. Furthermore, the predictors included in Cai et al.'s model¹⁰ were determined based on the relevant literature.

Statistical analysis

All statistical analyses were performed using R software (version 4.0.5). The mean (SD) is reported for quantitative variables, while the count (%) is reported for categorical variables. The R package compareGroups was used to analyze the differences in baseline data between the two cohorts. According to the latest risk assessment tool PROBAST, binary or multiclassification conversion of quantitative variables may lose significant information.²⁴ Therefore, this study did not perform binary or multiclassification transformations on quantitative variables, even though their relationship with the outcome variable may be nonlinear. Similarly, univariate analysis was not employed in this study because it did not take into account multivariate correction. Therefore, loss of some essential variables or finding spurious associations could occur.²⁴ Lasso regression is a reliable variable screening method. It performs a penalty regression on all variable coefficients so that relatively unimportant independent variable coefficients are gradually compressed to zero and thus excluded from the model.²⁵ Therefore, Lasso regression can address issues of variable collinearity while avoiding model overfitting and has apparent advantages in screening high-dimensional variables.²⁶ In this study, Lasso regression was used to initially screen variables, and then logistic regression was used to establish a multivariate model. We referred to this method as Lasso-logistic regression.¹² Finally, a predictive nomogram of DGE was developed based on the logistic model, and the polynomial formula was extracted by the R nomogram Ex package. The optimal Cutoff function was used to determine the thresholds for the two risk stratifications of the model.²⁷ Model similarity evaluation was performed using the likelihood ratio test. The Akaike information criterion (AIC) and Hosmer and Lemeshow test were used to evaluate the model fitting. The model performance was evaluated using the AUC and Misclass error. The net benefit of the model was evaluated using the decision curve analysis (DCA). The predictive ability of the model was evaluated using a clinical impact curve. p < 0.05 was considered to be statistically significant.

Model visualization

The code for data modeling and model visualization is available at https://github.com/ZongtingCode/DGERS.

Clinical research cohort features

The training cohort included 1251 PDAC patients who underwent PD in the ACS-NSQIP database from 2014 to 2017. The external validation cohort included 934 PDAC patients after PD in a Chinese NCC hospital from 2014 to 2019. The study cohort consists of PDAC cases originating from the pancreatic head or uncinate process. In the training cohort, there were 652 males and 599 females with an average age of 65.8 years and a mean BMI of 27.2 kg/m². In the external validation cohort, there were 496 males and 438 females, with an average age of 65.7 years and a mean BMI of 27.2 kg/m². Approximately 14.7% of patients in the training cohort (184/1251) developed DGE after surgery, whereas this rate was 15.0% (140/934) in the validation cohort. The proportion of clinically relevant DGE (grades B/C) associated with abdominal complications such as pancreatic fistula, bile leakage, bleeding, or sepsis/septic shock was 70.7% (130/184) in the training cohort and 72.9% (102/140) in the validation cohort. In both the training and validation cohorts, the median value of “days to pancreatic drain removal” was 6 (95% confidence interval: 2–26 and 2–25, respectively). The baseline characteristics of the two cohorts of PDAC patients treated with PD are summarized in Table 1. The two cohorts were well-matched. Intraoperative indicators, including surgical techniques and operation time, showed no significant differences between the cohorts (p > 0.05).

Lasso regression for variable screening

Given its effectiveness in handling large-scale variables, Lasso regression was employed in the training cohort for variable screening. As shown in Figure 2, the two vertical dashed lines in the chart represent the log λ of the minimum mean squared error (MSE; left dashed line) and the log λ of a standard error (SE) of the minimum distance (right dashed line). If there is an overfitting problem, then the distance from the minimum to the SE position is a very good starting point for solving the problem.²⁸ The model variables were reduced to 17 when log(λ) reached the minimum MSE, and 4 when it reached a standard error (1-fold SE) of the minimum distance, as determined by the Lasso internal cross-validation (10 k-fold; Figures 2a,b; Table S1). Although the AUC values of the model did not show significant differences between the two cutoffs (Figure 2c), we preferred to choose the point of 1-fold SE of the minimum distance (best λ value = 0.04), as it resulted in a simpler and more explanatory model with fewer variables. Four variables were ultimately selected using Lasso regression (Table 2): days to pancreatic drain removal, pancreatic fistula, sepsis/septic shock, and reoperation.

Development of a novel DGERS based on lasso-logistic regression

Next, we incorporated the four variables screened by Lasso regression into logistic regression for further modeling. The results showed that all four variables were identified as independent risk factors for DGE (p < 0.001). There was no evidence of multicollinearity or significant interaction effects among the remaining four variables (VIF <2; Table 3, Figure S2c,d), and the modeling was successful. We then developed a nomogram for predicting DGE based on Lasso-logistic regression (Figure 3a). The nomogram includes four predictive indicators: days to pancreatic drain removal, sepsis/septic shock, reoperation, and pancreatic fistula. Considering the limitations on the scale of the nomogram axis, we hardly obtained an accurate scale through the number of axes, especially for quantitative variables. Thus, we cannot calculate the accurate probability, which limits the wide application of the nomogram. Therefore, we further extracted the polynomial formula of the predictors to calculate the exact point of each indicator and the exact risk probability corresponding to the total points (Figure 3b). The risk of DGE was calculated using the following formula:

DGE risk = 1.42 × 10^−5 × (∑ Points) ^2–2.6 × 10^−8 × (∑ Points) ^3 + 7.37 × 10^−4 × (∑ Points) + 0.06.

where ∑ Points represents the sum of points based on relevant clinical factors.

To further stratify the risk of DGE, we applied the optimalCutoff function to calculate the binary risk stratification threshold in the training cohort. The optimal-cutoff value was determined to be 0.596. When the threshold was set to an integer value of 0.5, the model demonstrated good predictive ability within two specific subgroups (Figures 4a; S3a). Therefore, we initially set the threshold at 0.5 as the cutoff value to differentiate between the high-risk and low-risk groups. We named this cutoff value as the “max-cutoff” value. Subsequently, within the low-risk group, we identified an additional integer cutoff value, referred to as the “min-cutoff” value, to further stratify the original low-risk group into two subgroups. Remarkably, when the min-cutoff value was set to 0.3, the model displayed equally good predictive ability in these two specific subgroups (Figures 4b; S3b–d). Finally, we utilized the overlap of the two thresholds to create three risk stratifications: the low-risk, intermediate-risk, and high-risk groups. These groups corresponded to predicted probabilities of ≤0.3, >0.3 and ≤ 0.5, and >0.5, respectively. We refer to this risk stratification approach as the DGERS. The ROC curves of the models for each risk group in the training cohort demonstrated significant discriminative ability across all three risk groups. The AUC values for the low-, moderate-, and high-risk groups were 0.69, 0.63, and 0.74, respectively, with particularly prominent performance in the low-risk and high-risk groups (Figure 4b). Moreover, we conducted further analysis of the actual incidence of DGE in each stratified group, confirming the effectiveness of the DGERS method in the training cohort (Figure 4c; Table S2).

Evaluation and external validation of the novel model

The conventional method of variable screening typically involves univariate analysis or full variable inclusion. However, in this study, we employed a different approach by using Lasso-logistic regression for preliminary variable screening. Since univariate analysis is not recommended by the latest PROBAST as previously described,²⁴ we used the full variable inclusion method as a comparative model, referred to as the full-logistic model (Table S3). In addition, we also compared the DGE nomogram model reported by Cai et al.,¹⁰ which is referred to as Cai's logistic model. The analysis results (Table 4) showed that the above three models all had the predictive ability (p > 0.05, Hosmer and Lemeshow test; Figure 5a–c) in the validation cohort. There was a significant difference between the Lasso-logistic model and Cai's logistic model (p < 0.001, Likelihood ratio test). Compared with Cai's logistic model, the Lasso-logistic model had better model fitting (AIC, 955.31 vs. 1028.50), higher model performance (AUC, 73.4% vs. 59.2%), and there was no difference in the number of model variables and misclass error. Therefore, the Lasso-logistic model was superior to Cai's logistic model and the prediction efficiency of the model is acceptable. There was no significant difference between the Lasso-logistic model and the Full-logistic model (p = 0.416, Likelihood ratio test). However, the Lasso-logistic model had fewer independent variables (4 vs. 6) and slightly better predictive power than Full-logistic model (AUC, 73.4 vs. 71.8), so the Lasso-logistic model had better clinical operability and convenience (Table 4). These features are consistent with internal validation in the training cohort (Figure 4a–c). The DCA curve (Figure 6a) demonstrates that the net benefit of the Lasso-logistic model is significantly higher than that of Cai's logistic model in the threshold probability range of 0.18–0.93. However, there was no significant difference in net benefit between the Lasso-logistic model and the full logistic model. The relationship between the predicted DGE positive number and the actual positive number of the Lasso-logistic model at different threshold probabilities (with a cardinality of 1000) is depicted in Figure 6b. The Lasso-logistic model exhibits superior performance compared to the other two models in terms of predictive power and the number of variables. This indicates that the Lasso-logistic model has strong clinical applicability due to its improved accuracy and efficiency. To validate the discriminatory power of this risk stratification method, we plotted the ROC curves of the models for each risk group in the validation cohort. The results demonstrated a significant discriminative ability across all three risk groups. The AUC values for the low-, moderate-, and high-risk groups were 0.7, 0.61, and 0.74, respectively, with particularly strong performance observed in the low-risk and high-risk groups (Figure 6c). Similarly, the actual incidence of DGE in each stratified group confirmed the effectiveness of the DGERS method in the validation cohort (Figure 6d; Table S2).