Novel bleeding risk score for patients with atrial fibrillation on oral anticoagulants, including direct oral anticoagulants
Final decision: Sabine Eichinger, 12 January 2020.
ClinicalTrials.gov Identifier: NCT02105844.
Manuscript handled by: Sabine Eichinger
Funding Information
This study is funded by the Swiss Heart Foundation (grant to Prof. Rodondi). The Swiss-AF cohort study is supported by grants of the Swiss National Science Foundation (grant numbers 33CS30_148474 and 33CS30_177520).
Abstract
Objective
Balancing bleeding risk and stroke risk in patients with atrial fibrillation (AF) is a common challenge. Though several bleeding risk scores exist, most have not included patients on direct oral anticoagulants (DOACs). We aimed at developing a novel bleeding risk score for patients with AF on oral anticoagulants (OAC) including both vitamin K antagonists (VKA) and DOACs.
Methods
We included patients with AF on OACs from a prospective multicenter cohort study in Switzerland (SWISS-AF). The outcome was time to first bleeding. Bleeding events were defined as major or clinically relevant non-major bleeding. We used backward elimination to identify bleeding risk variables. We derived the score using a point score system based on the β-coefficients from the multivariable model. We used the Brier score for model calibration (<0.25 indicating good calibration), and Harrel's c-statistics for model discrimination.
Results
We included 2147 patients with AF on OAC (72.5% male, mean age 73.4 ± 8.2 years), of whom 1209 (56.3%) took DOACs. After a follow-up of 4.4 years, a total of 255 (11.9%) bleeding events occurred. After backward elimination, age > 75 years, history of cancer, prior major hemorrhage, and arterial hypertension remained in the final prediction model. The Brier score was 0.23 (95% confidence interval [CI] 0.19–0.27), the c-statistic at 12 months was 0.71 (95% CI 0.63–0.80).
Conclusion
In this prospective cohort study of AF patients and predominantly DOAC users, we successfully derived a bleeding risk prediction model with good calibration and discrimination.
Essentials
- Most current bleeding risk prediction tools for patients with atrial fibrillation (AF) and oral anticoagulants are not designed for patients on direct oral anticoagulants (DOACs), but DOACs have become a more and more popular choice of anticoagulant in AF patients.
- We present a new bleeding risk score derived from a prospective, population-based cohort of AF patients with predominantly DOAC users.
- Our score accurately identifies patients at low or high risk of bleeding after 1 year. After further external validation, this score will help the clinician to balance the risk of bleeding in AF patients, including DOAC users.
1 INTRODUCTION
Atrial fibrillation (AF), the most common arrhythmia, is associated with increased risk for cardiac thromboembolism.1 AF is present in approximately 1% to 2% of the population2 and associated with increased risk for cardiac thromboembolism,1 causing almost a third of all strokes.3 Thromboembolism and stroke risk can be greatly reduced if oral anticoagulants (OACs, including both vitamin K antagonists [VKAs] and direct oral anticoagulants [DOACs]) are administered, but this treatment increases bleeding risk.4, 5 Balancing bleeding risk against stroke risk for each patient is essential, but the clinical tools designed to predict a patient's risk of bleeding and thromboembolism are suboptimal.3
Bleeding risk scores like HAS-BLED,6 HEMORR2HAGES,7 ATRIA,8 and ORBIT9 were designed to identify patients at high risk of bleeding and to help doctors decide which patients can safely be given anticoagulants, but they showed limited predictive scores with c-statistics ranging from 0.54 to 0.61.10, 11 A few years ago, DOACs were introduced and have proven to be as effective as VKAs in preventing cardiac thromboembolism and stroke in AF patients, with lower bleeding risk. Though clinicians increasingly use DOACs in AF patients to prevent stroke and systemic embolisms,12, 13 all but two studies included very few DOAC users in their bleeding prediction models. The first of these studies, published in 2015, derived its ORBIT score based on patients on rivaroxaban in a large randomized trial (Rocket-AF) with strict inclusion and exclusion criteria and only patients who received a single DOAC.9 The other, a 2018 study by Rutherford et al., developed its score from a Norwegian patient registry14 that included patients on all types of DOAC, but their data source lacked prospective evaluation of the main outcome.
Given the limitations of existing scores and the need for better prediction tools for patients on DOAC, we developed and internally validated a novel clinical prediction score for patients with AF who were treated with either a VKA or DOAC based on data from a prospective cohort study with adjudicated clinical outcomes.
2 METHODS
We developed and internally validated a prognostic score for predicting bleeding in patients with AF under OAC treatment (VKA or DOAC) from the Swiss-Atrial Fibrillation (SWISS-AF) cohort study. SWISS-AF is a multicenter Swiss cohort study that includes patients aged ≥65 years with documented AF (paroxysmal, persistent, or permanent), already described in detail in Conen et al.15 The SWISS-AF study was approved by all local ethics committees (PB_2016-00793, for Bern, “Ethikkommission Nordwest- und Zentralschweiz” EKNZ 2014-067. KEK-BE Nr. 032/14); our study required no further review by an ethics committee. We excluded data from patients who were unable to provide informed consent, suffered only short episodes of reversible forms of AF, had had recent surgery (≤3 weeks prior to baseline), or were missing follow-up information. We also excluded patients who were not under OAC (VKA or DOAC) at baseline.
This study adheres to the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) statement.16 We internally validated the model we developed by applying dedicated methods in the development population.
2.1 Definition and assessment of outcomes
Our primary outcome was the time to major or clinically relevant non-major bleeding for up to 48 months after study inclusion. To better compare our model's scores to other bleeding risk scores, we focused on the score for prediction after 1 year. Our secondary endpoint analyses assessed the predictive accuracy of the score for major-, intracranial, and clinically relevant non-major bleeding. We drew our definition of major bleeding from the International Society on Thrombosis and Haemostasis: clinically overt fatal bleeding or bleeding that reduced haemoglobin level of ≥20 g/L within 7 days and required transfusion of at least two units of red blood cells, or symptomatic bleeding in a critical area or organ (intracranial, intraspinal, intraocular, pericardial, intra-auricular, intramuscular with compartment syndrome, retroperitoneal).17 Clinically relevant non-major bleeding was defined as bleeding that was not major, but was clinically overt and led to hospitalization, change of antithrombotic therapy, or necessitated a medical or surgical intervention.15
At each yearly study visit, patients were asked about bleeding events and their medical history was updated. If a patient had a bleeding event, local study nurses collected all relevant source documents, for example, hospital reports, laboratory results, operational reports. Local senior physicians then confirmed events and adjudicated the outcome based on the criteria for bleeding.
2.2 Statistical analysis
We calculated the proportion (%) and mean (± standard deviation [SD]) for all potential bleeding predictor candidates previously identified by literature search (Table S1 in supporting information) for continuous and dichotomous variables. Those variables were tested in univariable models for their association with the main bleeding endpoint. We used a ratio-likelihood test to check for linear association of continuous variables with the combined bleeding endpoint. To make it easier for clinicians to use the score, we chose the median for age as category cut-off and created our categories based on quartiles for variables that did not show linear association with the endpoint but had a normal distribution.
We analyzed time to first major or clinically relevant non-major bleeding event; non--bleeding-related deaths were a competing event. To do this, we used a maximum likelihood competing risk regression model, according to Fine and Gray's method18 entering those variables associated with P < .2 in univariable analyses. We used backward elimination to eliminate variables with a P-value >.05, so we could identify the remaining variables in the final prediction model. For variables with missing baseline data, we used multiple imputations19 based on all available full baseline datasets. We derived the risk score based on a point score system; we calculated the points we assigned to the predictors identified in the final model, by dividing each β-coefficient by the lowest β-coefficient and then rounding the result to the nearest integer.20 We divided patients into three categories of increasing bleeding risk (low, moderate, high). There are no generally accepted cut-offs for low- or high-risk categories, so we decided to use categories similar to those used by other scores: <3% for low bleeding risk; >6.4% as the cut-off for high bleeding risk.21 We calculated incidence rates with 95% confidence intervals (CI) for bleeding in each category, based on the observed bleeding events. We also applied the risk score to patients under either VKA or DOAC at baseline.
We assessed the overall discriminatory ability of the model and of the risk score with Harrel's c-statistic (summarized as the area under receiver operating characteristic curve [AUC ROC]) with 95% CI. The score's predictive accuracy was assessed at 6, 12, 24, 36, and 48 months as well as after the maximum follow-up (4.4 years). Model calibration was assessed with the Brier score,22 with <0.25 deemed good calibration. We also used the remaining predictors to calculate the ratio of expected/observed values.
For internal model and score validation, we used bootstrapping methods.23 We performed 500 bootstrap cycles in the original sample, resampling the same number of patients. First, we assessed apparent overall model discrimination within 500 bootstrap cycles. Next, we calculated shrinkage and optimism adjusted c-statistic for the score.
We used the same methods described above to assess c-statistics over time for existing bleeding risk scores designed for patients with AF (HAS-BLED,6 ATRIA,8 ORBIT9) and a new prediction score predicting bleeding in AF patients taking only DOACs.14 The scores were applied within our cohort for major and clinically relevant bleeding. We compared the performance of our score with previously existing scores’ performances at 12 months and 4.4 years by using DeLong et al.'s method.24 To make our results more comparable, we focused on the score's predictive performance at 12 months.
STATA Version 16.0. (Stata Corporation) was used for all statistical analyses.
3 RESULTS
The SWISS-AF Cohort study included 2415 patients with AF; of these 37 were lost to follow-up and 230 did not take OAC (VKA/DOAC) at baseline (Figure S1 in supporting information), which left 2147 patients on OAC. Patients’ baseline characteristics by bleeding status are presented in Table 1. Mean age was 73.4 (SD ± 8.2 years); 72.5% of our study population were men. During a mean follow-up of 2.1 years (maximum 4.4 years), there were 255 bleeding events, including 107 (42.0%) major bleeding events, of which 13 (12.2%) were intracranial. After 12 months, 25 major and clinically relevant bleedings occurred (2 intracranial), resulting in a 1.16% absolute bleeding risk at 1 year. The annual bleeding rate per person-year was 5.77% (95% CI 5.11%–6.53%) and 0.29% (95% CI 0.17%–0.51%) for intracranial bleedings.
| All (n = 2147) | Major & relevant bleeding (n = 255) |
No bleeding (n = 1892) |
|
|---|---|---|---|
| Age (years)a | 73.4 ± 8.2 | 73.1 ± 8.3 | 75.9 ± 8.2 |
| Sex (%) | |||
| Male | 1556 (72.5) | 1367 (72.3) | 189 (74.12) |
| Female | 591 (27.5) | 525 (27.7) | 66 (27.53) |
| Height (cm) | 172.2 ± 9.0 | 172.1 ± 8.6 | 172.2 ± 9.0 |
| Weight (kg) | 82.6 ± 16.4 | 82.6 ± 16.4 | 82.6 ± 15.9 |
| Type of AF (%)b | |||
| Paroxysmal | 919 (42.8) | 106 (41.6) | 813 (43.0) |
| Persistent | 652 (30.4) | 70 (27.5) | 582 (30.8) |
| Permanent | 576 (26.8) | 79 (31.0) | 497 (26.2) |
| Smoking (%)c | |||
| Active | 145 (6.8%) | 15 (5.88) | 130 (6.88) |
| Former smoker | 1062 (49.5) | 118 (46.27) | 944 (49.97) |
| Never smoker | 937 (43.6%) | 122 (47.84) | 815 (43.14) |
| Type of VKA (%)d | 938 (43.7) | 126 | 812 |
| Phenprocoumon | 725 (33.8) | 113 (44.3) | 612 (32.4) |
| Acenocoumarol | 213 (9.9) | 13 (5.1) | 200 (10.6) |
| DOAC (%)d | 1209 (56.3) | 129 (50.6) | 1080 (57.1) |
| Rivaroxaban | 886 (41.3) | 88 (34.5) | 798 (42.2) |
| Dabigatran | 79 (3.7) | 11 (4.3) | 68 (3.6) |
| Apixaban | 204 (9.5) | 27 (10.6) | 177 (9.4) |
| Edoxaban | 40 (1.9) | 3 (1.2) | 37 (2.0) |
| Poor INR control (%)e | 89 (4.15) | 13 (5.1) | 76 (4.0) |
| Prior stroke/TIA (%)f | 443 (20.6) | 67 (26.3) | 376 (19.9) |
| Heart failure (%) | 569 (26.5) | 83 (32.6) | 486 (25.7) |
| Hypertension (%)g | 1516 (70.6) | 209 (82.0) | 1307 (69.1) |
| Hemoglobin (g/L)h | 135.5 ± 19.1 | 131.6 ± 19.4 | 136.1 ± 19.0 |
| Hematocrit (L/L)h | 40.4 ± 5.4 | 39.6 ± 5.4 | 40.6 ± 5.4 |
| Thrombocytes (G/L)h | 223.4 ± 73.2 | 216.1 ± 71.4 | 224.4 ± 73.4 |
| History of cancer (%)i | 339 (15.8) | 54 (21.2) | 285 (15.1) |
| Prior major haemorrhage (%) | 48 (2.24) | 10 (3.9) | 38 (2.0) |
| Diabetes (%) | 373 (17.4) | 38 (14.9) | 335 (17.7) |
| History of falls (%) | 179 (8.3) | 29 (11.4) | 150 (7.9) |
| Prior gastric ulcer (%) | 93 (4.3) | 17 (6.7) | 76 (4.0) |
| Coronary artery disease (%) | 351 (16.4) | 46 (18.0) | 305 (16.1) |
| Other embolic events (%) | 112 (5,2) | 16 (6.3) | 96 (5.1) |
| History of VTE (%) | 205 (0.6) | 23 (9.0) | 182 (9.6) |
| Antiplatelet therapy (%) | 340 (15.8) | 46 (18.0) | 294 (15.6) |
| Use of NSAID (%) | 47 (2.2) | 8 (3.1) | 39 (2.1) |
| Use of PPI (%)j | 664 (30.9) | 97 (38.0) | 567 (30.0) |
| Peripheral arterial disease (%) | 170 (7.9) | 27 (10.6) | 143 (7.6) |
| Risky alcohol consumption (%)k | 87 (4.1) | 10 (3.9) | 77 (4.1) |
| ALAT (U/L)l | 23.6 ± 10.8 | 23.0 ± 9.7 | 23.7 ± 10.9 |
| Creatinine (μmol/L)m | 110.5 ± 51.6 | 119 ± 55.6 | 109 ± 51.0 |
- Abbreviations: AF, atrial fibrillation; ALAT, alanine aminotransferase; DOAC, direct oral anticoagulant; INR, international normalized ratio; NSAID, non-steroidal anti-inflammatory; PPI, proton pump inhibitor; TIA, transient ischemic attack; VKA, vitamin K antagonist; VTE, venous thromboembolism.
- Definition of variables:
- a Age: age in years at study inclusion.
- b Type of AF: paroxysmal, self-terminating AF lasting <7 days without need for cardioversion, documented at least twice within 60 months; persistent AF, AF that lasted 7 days or longer and/or requiring cardioversion documented in the last 60 months by ECG or rhythm monitoring devices; permanent AF, AF lasts permanently, cardioversion has failed or not been attempted.
- c Smoking as assessed by self-report.
- d Direct oral anticoagulants: type of anticoagulant at baseline.
- e Poor INR control: <30% if INR values in therapeutic range.21
- f Prior stroke/TIA: history of ischemic or hemorrhagic stroke or TIA before study inclusion, self-reported or from available medical documentation.
- g Hypertension: history of hypertension, self-reported or from available medical documentation, or taking oral antihypertensives, controlled or uncontrolled.
- h Hemoglobin, hematocrit, and thrombocytes: measured within the last 6 months prior to study inclusion.
- i History of cancer: any active or cured cancer.
- j Use of proton pump inhibitors at baseline.
- k Risky alcohol consumption: >1 standard glass/d (SG) for women, >2 SG/d for men.31
- l ALAT: in U/L, measured at baseline.
- m Creatinine: in mmol/L, measured at baseline.
3.1 Potential predictors
From the literature we identified 28 risk factors with reported independent association with bleeding (Table S1). The SWISS-AF study collected most of those variables at baseline. Table 2 shows the final predictors we entered into the model. Unlike earlier prediction models, ours did not find history of diabetes mellitus was a risk factor in the univariate analysis, so we did not consider it as a predictor for the combined bleeding endpoint.
| Variable | Univariable | Multivariable analysis | ||||
|---|---|---|---|---|---|---|
| Sub-hazard ratio | β-coeff. (95% CI) | p-value | Sub-hazard ratio | β-coeff. (95% CI) | p-value | |
| Age ≥ 75 | 1.76 (1.38–2.25) | 0.57 (0.32–0.81) | <.001 | 1.61 (1.25–2.07) | 0.48 (0.22; 0.73) | <.001 |
| Hypertension | 1.79 (1.30–2.46) | 0.58 (0.26; 0.90) | <.001 | 1.62 (1.17–2.25) | 0.48 (0.16; 0.81) | .004 |
| History of cancer | 1.57 (1.18–2.10) | 0.45 (0.16; 0.74) | .002 | 1.41 (1.05–1.88) | 0.34 (0.05; 0.63) | .021 |
| Prior major hemorrhage | 2.07 (1.14–3.72) | 0.73 (0.14; 1.32) | .016 | 2.03 (1.12–3.67) | 0.70 (0.11; 0.13) | .020 |
| Prior stroke/TIA | 1.36 (1.03–1.80) | 0.31 (0.03; 0.59) | .028 | |||
| Use of PPI | 1.35 (1.05–1.74) | 0.30 (0.05; 0.55) | .020 | |||
| History of falls | 1.93 (1.30–2.90) | 0.66 (0.27; 1.05) | .001 | |||
| Creatinine (quartiles) | ||||||
| 2 | 0.84 (0.58– 1.22) | −0.17 (−0.54; 0.20) | .367 | |||
| 3 | 1.16 (0.80–1.66) | 0.15 (−0.22; 0.51) | .434 | |||
| 4 | 1.56 (1.13–2.20) | 0.46 (0.12; 0.79) | .007 | |||
| Peripheral arterial disease (PAD) | 1.49 (0.99–2.24) | 0.40 (−0.01; 0.81) | .053 | |||
| Hemoglobin (quartiles) | ||||||
| 2 | 0.95 (0.66–1.37) | −0.48 (−0.41–0.32) | .798 | |||
| 3 | 0.72 (0.48–1.06) | −0.33 (−0.73–0.59) | .096 | |||
| 4 | 0.67 (0.44–1.04) | −0.39 (−0.82–0.42) | .077 | |||
| Hematocrit | 0.98 (0.95–1.01) | −0.02 (−0.52; 0.01) | .115 | |||
| Smoking | 0.85 (0.69–1.05) | −0.16 (−0.37; 0.05) | .131 | |||
| Antiplatelet therapy | 1.24 (0.91–1.70) | 0.22 (−0.10; 0.53) | .171 | |||
| Diabetes | 0.80 (0.56–1.11) | −0.24 (−0.58; 0.11) | .179 | |||
| ALAT | 0.99 (0.98–1.00) | −0.01 (−0.02; 0.01) | .284 | |||
| Prior gastric ulcer | 1.27 (0.77–2.10) | 0.24 (−0.27; 0.74) | .356 | |||
| Coronary artery disease | 1.16 (0.84–1.60) | 0.15 (−0.17; 0.47) | .362 | |||
| Heart failure | 1.12 (0.86–1.47) | 0.12 (−0.15; 0.39) | .402 | |||
| Use of NSAID | 1.36 (0.62–2.98) | 0.31 (−0.47; 1.09) | .437 | |||
| Female sex | 0.94 (0.71–1.24) | −0.06 (−0.34; 0.22) | .678 | |||
| History of VTE | 1.06 (0.69–1.64) | 0.06 (−0.38; 0.50) | .783 | |||
| Thrombocytes | 1.00 (0.99–1.00) | 0.00 (0.00; 0.00) | .786 | |||
| Body mass index | 1.00 (0.98–1.03) | 0.00 (−0.02; 0.03) | .796 | |||
| Risky alcohol consumption | 1.06 (0.58–1.95) | 0.06 (−0.55; 0.67) | .85 | |||
| Type of atrial fibrillation | 1.01 (0.89–1.17) | 0.02 (−0.13; 0.16) | .854 | |||
| Poor INR control | 1.01 (0.60–1.70) | 0.01 (−0.51; 0.53) | .969 | |||
| Treatment with VKA | 1.00 (0.79–1.28) | 0.00 (−0.24; 0.25) | .988 | |||
| Treatment with DOAC | 1.00 (0.78–1.27) | 0.00 (−0.25; 0.24) | .988 | |||
- Abbreviations: AF, atrial fibrillation; ALAT, alanine aminotransferase; CI, confidence interval; DOAC, direct oral anticoagulant; INR, international normalized ratio; NSAID, non-steroidal anti-inflammatory; PPI, proton pump inhibitor; TIA, transient ischemic attack; VKA, vitamin K antagonist; VTE, venous thromboembolism.
After a test for linearity, no continuous variables showed a linear association with the combined endpoint, so all continuous variables were categorized. After multivariable competing risk analysis and stepwise backward selection, age ≥ 75 years, history of cancer, arterial hypertension, and history of major bleeding were retained in the final prediction model. In a sensitivity analysis, adding NSAR, sex, and use of aspirin at baseline in the multivariable model and repeating the multivariable stepwise backwards analysis, the same four variables were identified.
3.2 Score derivation, model calibration, and discrimination
We assigned point scores based on the β-coefficient from the prediction model using a point score system (Table 3).20 The Brier score was 0.23 (95% CI 0.19–0.27), showing that the model was well calibrated; expected/observed probabilities were 1.02 at 12 months, 0.99 at 24 months, and 0.99 and 36 months (Figures S1–S3 in supporting information). After building the score from the prediction model's β-coefficients, the c-statistic for the score was 0.71 (95% CI 0.63–0.80) at 12 months. The predictive ability of the score decreased over time, from 0.66 (95% CI 0.61–0.72) at 24 months to 0.64 (95% CI 0.60–0.68) at 36 and 48 months. For the whole follow-up period, the score's c-statistic was 0.62 (95% CI 0.59–0.65; Table 4).
| β-coefficient (95% CI) | Points assigned | |
|---|---|---|
| Age ≥75 years | 0.476 (0.223 −0.730) | 1.5 |
| Hypertension | 0.483 (0.157– 0.810) | 1.5 |
| History of cancer | 0.341 (0.051– 0.631) | 1 |
| Prior major bleeding | 0.707 (0.112– 1.301) | 2 |
| Max points: | 6 |
- Abbreviation: CI, confidence interval.
| No events/no patients |
Our score C-statistics (95% CI) |
HAS-BLED | ATRIA | ORBIT | Rutherford score | |
|---|---|---|---|---|---|---|
| Up to 6 months | 3/2147 | 0.79 (0.78–0.80) | 0.70 (0.45–0.94) | 0.74 (0.62–0.86) | 0.59 (0.46–0.72) | 0.75 (0.43–1.06) |
| Up to 12 months | 25/2147 | 0.71 (0.63–0.80) | 0.63 (0.52–0.74) | 0.73 (0.66–0.80) | 0.69 (0.60–0.78) | 0.72 (0.63–0.82) |
| Up to 24 months | 90/2147 | 0.66 (0.61–0.72) | 0.60 (0.54–0.65) | 0.66 (0.61–0.71) | 0.64 (0.58–0.69) | 0.67 (0.61–0.72) |
| Up to 36 months | 172/2147 | 0.64 (0.60–0.68) | 0.61 (0.56–0.65) | 0.65 (0.61–0.69) | 0.63 (0.59–0.67) | 0.66 (0.61–0.70) |
| Up to 48 months | 233/2147 | 0.64 (0.60–0.68) | 0.60 (0.56–0.64) | 0.64 (0.60–0.68) | 0.62 (0.58–0.66) | 0.65 (0.61–0.69) |
| Up to 4.4 years | 255/2147 | 0.62 (0.59–0.65) | 0.60 (0.56–0.63) | 0.60 (0.57–0.64) | 0.59 (0.55–0.62) | 0.62 (0.58–0.66) |
- Abbreviation: CI, confidence interval.
When we stratified bleeding risk into three categories (low, moderate, high), most patients (n = 1579, 73.5%) were classed as moderate; there were 5.9 bleeding events (95% CI 5.1–6.8) per 100 patient years. Overall, 394 (18.4%) of patients were at low risk of bleeding (2.5 bleedings per 100 years) and 174 (8.1%) were at high risk (12.9 per 100 patient years; Table 5).
| Risk category (score points) | Risk category distribution | Incidence of bleeding from derivation | Incidence of bleeding after validation | |
|---|---|---|---|---|
| n (%) | n per 100 patient years (95% CI) | per 100 patient years (95% CI) | ||
| Low (0–1) | 394 (18.4) | 21 | 2.5 (1.6–3.8) | 2.5 (1.0–4.1) |
| Moderate (1.5–3) | 1579 (73.5) | 190 | 5.9 (5.1– 6.8) | 5.9 (4.7–6.9) |
| High (>3) | 174 (8.1) | 44 | 12.9 (9.7–17.4) | 12.9 (8.8–17.4) |
- Abbreviation: CI, confidence interval.
3.3 Validation
The performance of the score after internal validation showed an AUC of 0.62 (95% CI 0.59–0.66) taking into account the entire follow-up period. These results were similar to those from the derivation. Optimism-adjusted c-statistic was 0.64 for the score.
3.4 Comparison with existing bleeding risk scores
We compared predictive performance at 12 months for existing bleeding risk scores and found the prediction accuracy of the ATRIA and Rutherford scores was very similar to ours. Our score was more accurate than HAS-BLED and ORBIT scores after 12 months (Table 4), but differences did not show statistical significance (our score 0.71 [95% CI 0.63–0.80] vs. HAS-BLED 0.63 [95% CI 0.52–0.74] P = .28, our score vs. ORBIT 0.69 [95% CI 0.60–0.78] P = .76, Table S2 in supporting information).
Considering the full follow-up period, all scores predicted bleeding risk about equally well (ATRIA 0.60 [95% CI 0.57–0.64], HAS-BLED 0.60 [95% CI 0.56–0.63], ORBIT 0.59 [95% CI 0.55–0.62], and the score from Rutherford et al. 0.62 [95% CI 0.58–0.66]; Tables 4 and S2).
3.5 Secondary analyses
The discriminative ability of our score for major bleeding was 0.67 (95% CI 0.53–0.81) up to 1 year, ranging to 0.62 (95% CI 0.57–0.68) after the whole follow-up period. C-statistics for clinically relevant non-major bleeding was 0.60 (95% CI, 0.54–0.66) up to 12 months and 0.61 (95% CI 0.56–0.65) after the entire follow-up period.
For intracranial bleeding, the c-statistic was 0.63 (95% CI 0.51−0.75) for the entire duration (4.4 years); analyses were limited by the low number of such events (n = 2 after 12 months and n = 13 for the overall follow-up). When applied to patients treated with only DOACs, the c-statistic for our score was 0.73 (95% CI 0.59–0.87) at 12 months, and 0.64 (95% CI 0.59–0.69) for the whole follow-up period. The c-statistic for the score that estimated combined bleeding endpoint for patients only given VKA was 0.58 (0.29–0.87) after 12 months and 0.59 (95% CI 0.54–0.64) after the entire follow-up period.
4 DISCUSSION
Based on a Swiss multi-center prospective cohort study, we developed a clinical prediction model with good calibration and good discrimination for major and clinically relevant moderate-to-minor bleeding in patients with AF who took oral anticoagulants (VKAs or DOACs); c-statistics ranged from 0.76 at 6 months to 0.62 at 4.4 years. More than 50% of patients in the cohort were treated with various DOACs. To our knowledge, this is the first bleeding prediction model from a prospective cohort study including a considerably high proportion of DOACs in patients with AF.
When we compared the performance of established scores in our cohort, our results aligned with those in the literature.11 Only the Rutherford et al. and ORBIT bleeding risk prediction scores were derived to predict bleeding in patients on DOACs; the Rutherford et al. score was derived to predict bleeding in patients who received DOACs only, while ORBIT was developed to predict bleeding in patients on rivaroxaban and VKAs. Our score included patients who used VKAs and varying DOACs, so it may be more representative for patients with AF seen in clinical practice. Our and Rutherford et al.'s scores made similarly accurate predictions after 1 year when we applied them to our cohort: c-statistics were 0.71 for our score (95% CI 0.63–0.80), and 0.72 for Rutherford et al. (95% CI 0.63–0.82) after 1 year; for the whole follow-up period, the scores were almost the same (ours was 0.62, 95% CI 0.59–0.65; Rutherford et al.’s was 0.62, 95% CI 0.58–0.66).14 But the Rutherford et al. score was derived from a retrospective population study, which is not a recommended method for deriving a prediction model.25 A study from a Danish registry examined the predictive accuracy of the HAS-BLED, ATRIA, and ORBIT scores for major bleeding after 1 year in patients with AF who used DOACs; their study found comparable c-statistics to those we did for overall follow-up (ATRIA 0.59, 95% CI 0.57–0.60; HAS-BLED 0.58, 95% CI 0.57–0.59; ORBIT 0.61, 95% CI 0.59–0.62).26
Our score better predicted bleeding for patients only taking DOACs, which suggests that DOACs’ and VKAs’ different pharmacological effects require that we assess patients who take each of these drugs differently. In our cohort, patients using VKA were older and were suffering more from arterial hypertension and chronic kidney disease. More bleedings occurred in VKA users (13% of VKA users had major and clinically relevant non-major bleeding within the entire follow-up and bleedings occurred in 10% of DOAC users). These findings align well with previous trials, showing difference between VKAs and DOACs.27 We did not derive a prediction model for the patients who only used DOACs because our statistical power was too limited.
Most scores were derived to predict major bleeding over 1 or 2 years. Our score assessed a combined bleeding endpoint up to 4.4 years of follow-up. Over the long term (after 6.5 years), HAS-BLED’s c-statistic was 0.58 for major bleeding.28 A study that evaluated Thrombolysis in Myocardian Infarction (TIMI) score-significant bleedings (defined as major or minor) and bleeding requiring medical attention in patients using OACs and antiplatelet therapy, with a 3-year follow-up found AUC was 0.62 for HAS-BLED and 0.61 for ORBIT.29 As in our study, the occurrence of more confounding variables, like starting to take aspirin, age, and comorbid conditions, might account for the decrease in discriminative power over time.
Although c-statistics are not excellent and do decrease over time, our score may better identify patients at the extremes of low and high risk of bleeding; if so, it could help clinicians weigh the risks and benefits of OACs.
Similar to current AF guidelines30 not suggesting to withhold OACs for patients with a high risk of bleeding, our study cannot answer the question when to withhold OAC treatment. Current AF guidelines30 suggest the HAS-BLED score as a risk assessment tool to try to reduce bleeding risk by treating obvious risk factors (e.g., hypertension).
4.1 Strengths and limitations
An important strength of this study was that the analyzed data were from a large, prospective cohort study with broad inclusion criteria and thus broad external validity. By combining a literature review with a prospective analysis of associated risk factors, we found well-supported predictors for a stable model. Another strength of our study is the validation of known bleeding risk scores in this cohort of predominantly DOAC users. This study had several limitations to consider. The most important limitation is the lack of external validation. We used bootstrapping methods for internal validation as a split sample method would not have allowed for sufficient power to derive the score. In contrast to the HAS-BLED score, our score mostly consists of variables for risk factors that cannot be modified to reduce bleeding risk.
However, our score may assist with the identification of patients who may benefit from more frequent clinical monitoring (e.g., to assess for signs of occult bleeding or the need for dose adaptation of DOACs in cases of concomitant renal dysfunction). Another limitation is the definition of history of cancer, which encompassed active or cured cancer of any type. A definition limited to active cancer might have led to a stronger association between cancer and bleeding (potentially “non-differential” misclassification). However, even with this broad definition of cancer, it remains an independent predictor in our score.
Also, the patients in this bleeding risk model were mostly elderly. Therefore, the predictive ability in younger patients remains unknown.
5 CONCLUSION
In this prospective cohort study of patients with AF we derived a bleeding risk prediction model with good calibration and discrimination at 1 year. Our score identifies patients at low risk of bleeding who can safely use and benefit from anticoagulants, and those at high risk, whose risk of anticoagulation should be carefully evaluated after controlling for all known bleeding risk factors, but it should be externally validated before being implemented into practice.
ACKNOWLEDGMENT
We thank Kali Tal for her editorial assistance.
CONFLICTS OF INTEREST
DC holds a McMaster University Department of Medicine Mid-Career Research Award. His work is supported by the Hamilton Health Sciences RFA Strategic Initiative Program. LHB has received research grants from the Swiss National Science Foundation, the Swiss Heart Foundation, and the University of Basel. He has received an unrestricted research grant from AstraZeneca, as well as consultancy and advisory board fees from Amgen, Bayer, Bristol-Myers Squibb, and Claret Medical. MK reports personal fees from Bayer, Böhringer Ingelheim, Pfizer BMS, Daiichi Sankyo, Medtronic, Biotronik, Boston Scientific, and Johnson & Johnson; grants from Bayer, Pfizer, Boston Scientific, BMS, and Biotronik. The other authors do not declare a conflict of interest.
AUTHOR CONTRIBUTIONS
Conception and design of the study: MF, JD, DS, RL, DC, DA, NR. Acquisition of data: LA, MF, DS, CF, LR, UF, SA, GM, JS, DS, PA, RK, MS, MK, LHB, JB, SO, DC, DA, NR. Analysis and interpretation of data: LA, MF, LS, CDG, JD, CB, DA, NR. Drafting of the article: LA, MF, CGD, CB, NR. Acquisition of funding: DS, JD, MR. Critical revision for scientific content: all listed authors. Final approval of the version to be submitted: LA, MF, NR.
