The National Early Warning Score on admission predicts severe disease and in-hospital mortality of the coronavirus disease 2019 Delta variant: A retrospective cohort study
Abstract
Background
Clinical risk scores are widely used in emergency medicine, and some studies have evaluated their use in patients with coronavirus disease 2019 (COVID-19). However, no studies have evaluated their use in patients with the COVID-19 Delta variant. We aimed to study the performance of four different clinical scores (National Early Warning Score [NEWS], quick Sequential Organ Failure Assessment [qSOFA], Confusion, Respiratory rate, Blood pressure, and Age ≥65 [CRB-65], and Kanagawa score) in predicting the risk of severe disease (defined as the need for intubation and in-hospital mortality) in patients with the COVID-19 Delta variant.
Methods
This was a retrospective cohort study of patients hospitalized with suspected severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) Delta variant infection between June 1 and December 31, 2021. The primary outcomes were the sensitivity and specificity of the aforementioned clinical risk scores at admission to predict severe disease. Areas under the receiver operating characteristic curves (AUROCs) were compared between the clinical risk scores and we identified new cut-off points for all four scores.
Results
A total of 249 adult patients were included, of whom 18 developed severe disease. A NEWS ≥7 at admission predicted severe disease with 72.2% sensitivity and 86.2% specificity. The NEWS (AUROC 0.88) was superior to both the qSOFA (AUROC 0.74) and the CRB-65 (AUROC 0.67), and there was no significant difference between the NEWS and Kanagawa score (AUROC 0.86).
Conclusion
The NEWS at hospital admission predicted the severity of the COVID-19 Delta variant with high accuracy.
BACKGROUND
The coronavirus disease 2019 (COVID-19) pandemic threatens health care systems worldwide. COVID-19, the illness caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), emerged in late 2019 and spread very quickly across the globe. Within the first 2 years of the COVID-19 pandemic, more than 450 million cases were reported worldwide.1 In Japan, the first case of COVID-19 was reported on January 2020, and the cumulative incidence of reported cases has now reached more than 19.1 million confirmed cases and more than 40,198 deaths.2
In April 2021, the B.1.617.2 variant, known now as the Delta variant, was designated as a variant of concern by the World Health Organization.3 Soon after, the Delta variant emerged in Japan as well and eventually replaced the existing strains. By August 2021, the medical infrastructure was strained and hospital capacity was overwhelmed in Japan.4, 5 Appropriate clinical scores needed to be identified to risk stratify patients with COVID-19 and identify critically ill patients at that time.
Traditional clinical risk stratification tools are widely used for triage and continuous assessment of patients with severe infectious diseases. Several studies have evaluated the use of these tools in patients with SARS-CoV-2 infection with variable results.6-11 Furthermore, it has been confirmed that the severity prediction ability of the National Early Warning Score (NEWS)6-8 and NEWS29, 10 is significant.
The Delta variant has been reported to be more severe than any previous variants.12-14 Wise14 showed that the Delta variant doubles the risk of hospital admission compared with the Alpha variant. Given that the Delta variant has a different clinical course, it is necessary to investigate the performance of clinical risk scores at admission to predict outcomes in patients hospitalized with suspected SARS-CoV-2 Delta variant infection.
Therefore, we investigated the performance of four different clinical scores in patients admitted to the hospital with suspected Delta variant infection in predicting severe disease. We also aimed to identify other cut-off points to improve the detection of severe disease in this population. Three of them are world-famous triage scores, and one of them is a score originally developed by Kanagawa prefecture in Japan.
METHODS
The aim of this study was to assess and compare the performance of NEWS and other clinical risk scores at admission to predict the severity and in-hospital mortality in patients with suspected COVID-19 Delta variant infection.
This is a retrospective cohort study carried out at International University of Health and Welfare Narita Hospital in Chiba prefecture. The hospital was opened in March 2020 and treated approximately 1200 hospitalized patients with COVID-19 at the time of the study.
Study population
Patients admitted to the hospital with suspected COVID-19 Delta variant infection from June 1 to December 31, 2021, were consecutively included in the study. This study does not include strains other than the Delta variant. COVID-19 was confirmed by qualitative detection of nucleic acid from SARS-CoV-2 in throat or nasal secretions by use of real-time polymerase chain reaction.15 The Delta variant emerged in June 2021 and gradually replaced existing strains. Then, for this period only, it was verified to see whether it was the Delta variant by qualitative detection of nucleic acid from SARS-CoV-2 B.1.617.2.16 From August to December 2021, a complete replacement by the delta variant occurred, and we judged that all patients with COVID-19 were infected with the Delta variant. The following patients were excluded: (1) those aged <16 years and (2) patients transferred from another hospital.
Measures and definitions
Basic characteristics of enrolled patients were collected from the medical charts, including relevant demographic and comorbidities, as well as data used for clinical scores. Vitals signs included were those first recorded after arriving at the hospital.
The NEWS is a standardized clinical scoring system developed to improve detection of deterioration in acutely ill patients (Appendix S1).17 It is based on aggregate scoring of six physiological parameters: respiratory rate, oxygen saturation, systolic blood pressure, pulse rate, level of consciousness or new confusion, and body temperature. A NEWS of 7 is considered a key score that may indicate clinical deterioration and should prompt urgent response by a clinician or a team with competence in assessment and treatment of acutely ill patients. We have reported the usefulness of NEWS as a prehospital triage tool in Japan.18
The Quick Sequential Organ Failure Assessment (qSOFA) has been recommended as the tool of choice to assess organ dysfunction in patients with suspected sepsis.19 Three clinical variables (altered mental status, systolic blood pressure ≤100 mmHg, and respiratory rate of ≥22/min) are scored with 1 point each. A qSOFA score ≥2 should prompt clinicians to investigate for organ dysfunction, initiate or escalate therapy, and to consider increased monitoring or referral to an intensive care unit.
The Confusion, Respiratory rate, Blood pressure, and Age ≥65 (CRB-65) is a clinical score developed for risk stratification of patients with community-acquired pneumonia. One point each is given for the following clinical variables: new confusion, respiratory rate ≥30, and systolic blood pressure <90 mmHg or diastolic blood pressure ≤60 mmHg. In addition, age ≥65 years is scored with 1 point. A score of 3 or more indicates a need for hospitalization and in-patient management.20
We also examined the Kanagawa admission priority assessment score (version 3), which was updated on August 17, 2021, a clinical score developed for risk stratification of patients with COVID-19 in Kanagawa prefecture.21 Points are given for gender, age, comorbidities, clinical presentation, and X-ray/computed tomography features (Appendix S2). A total score of 5 points or more is considered appropriate for hospitalization. However, the validity of this score has not yet been disclosed by Kanagawa prefecture.
We defined severe disease as a composite measure of intubation treatment or death during hospitalization for any reason during the hospital stay. In-hospital mortality was defined as death during hospital stay for any reason.
Statistical analysis
Continuous variables are presented as the mean ± standard deviation, and categorical variables as the number (%). We used the Mann–Whitney test for continuous variables and the chi-square test for categorical variables to compare the characteristics of severe versus nonsevere diseases.
To assess the discriminatory ability of clinical tools to predict severe disease, we calculated sensitivity, specificity, positive predictive value, and negative predictive value with 95% confidence intervals (CIs) using previously validated cut-off values: NEWS ≥7, qSOFA score ≥2, CRB-65 score ≥3, and Kanagawa score ≥5. By varying cut-off values, we calculated discrimination measures and Youden's index, which is defined as a difference between sensitivity and 1–specificity. To summarize the performance of the risk stratification tool, the receiver operating characteristic curves for the clinical risk scores were depicted and the area under the receiver operating characteristic curves were compared using DeLong's test implemented in MEDCALC statistical software (http://www.medcalc.org). All other statistical analyses were conducted using SPSS version 25 (IBM, Chicago, IL, USA).
RESULTS
From June 1 to December 31, 2021, 249 adult patients were included in the study. The study included 182 males with mean age of 48.4 ± 13.2 years. Eighteen patients (7.2%) had the outcome of severe disease: all 18 patients were intubated, and 3 patients died during hospitalization. Baseline characteristics of enrolled patients are summarized in Table 1, divided by disease severity.
| Demographics | All (n = 249) | Severe disease (n = 18) | Nonsevere disease (n = 231) | p value |
|---|---|---|---|---|
| Age (years), mean (SD) | 48.4 (13.2) | 56.0 (9.9) | 47.8 (13.2) | <0.01 |
| Male gender, n (%) | 182 (73.1) | 15 (83.3) | 167 (72.3) | 0.41 |
| Body mass index (kg/m2), mean (SD) | 25.4 (5.2) | 28.3 (9.8) | 25.2 (4.6) | 0.03 |
| Clinical presentation | ||||
| CT % specific to COVID-19, mean (SD) | 21.0 (18.4) | 45.0 (16.2) | 19.2 (17.3) | <0.01 |
| Comorbidities | ||||
| Hypertension, n (%) | 48 (19.3) | 6 (33.3) | 42 (18.2) | 0.13 |
| Diabetes mellitus, n (%) | 47 (18.9) | 8 (44.4) | 39 (16.9) | <0.01 |
| Hyperlipidemia, n (%) | 30 (12.0) | 4 (22.2) | 26 (11.3) | 0.25 |
| Malignancy, n (%) | 6 (2.4) | 0 (0) | 6 (2.6) | >0.99 |
| Coronary disease, n (%) | 8 (3.2) | 2 (11.1) | 6 (2.6) | 0.107 |
| Chronic kidney disease, n (%) | 2 (0.8) | 0 (0) | 2 (0.9) | >0.99 |
| Dialysis, n (%) | 0 (0) | 0 (0) | 0 (0) | NA |
| Chronic respiratory disease (including asthma), n (%) | 2 (0.8) | 0 (0) | 2 (0.9) | >0.99 |
| Cirrhosis, n (%) | 2 (0.8) | 0 (0) | 2 (0.9) | >0.99 |
| Immunosuppressed state, n (%) | 2 (0.8) | 0 (0) | 2 (0.9) | >0.99 |
| Pregnant, n (%) | 0 (0) | 0 (0) | 0 (0) | NA |
- Abbreviations: COVID, coronavirus disease 2019; CT, computed tomography; NA, not applicable; SD, standard deviation.
In total, 45 patients (18.0%) had a NEWS ≥7 at admission; 12 patients (4.8%) presented with qSOFA ≥2, 9 patients (3.6%) presented with CRB-65 ≥2, and 199 (79.9%) patients presented with Kanagawa score ≥5. Table 2 shows vital signs and clinical risk scores at admission by disease severity. In the severe group, tachypnea, tachycardia, and decreased oxygen saturation were more serious.
| Parameters | All (n = 249) | Severe disease (n = 18) | Non-severe disease (n = 231) | p value |
|---|---|---|---|---|
| Mean (SD) | Mean (SD) | Mean (SD) | ||
| Vital signs | ||||
| Body temperature (°C) | 37.2 (0.9) | 37.6 (0.9) | 37.2 (0.9) | 0.215 |
| Respiratory rate (breaths/min) | 20.8 (3.9) | 26.7 (4.8) | 20.4 (3.4) | 0.000 |
| Oxygen saturation (%) | 95.8 (2.3) | 93.2 (3.3) | 96.1 (2.1) | 0.003 |
| Heart rate (beats/min) | 90.3 (16.7) | 100.1 (17.0) | 89.6 (16.5) | 0.004 |
| Systolic blood pressure (mmHg) | 121.0 (18.5) | 124.8 (25.1) | 120.1 (18.0) | 0.431 |
| Clinical risk scores | ||||
| NEWS | 3.83 (2.75) | 7.72 (2.42) | 3.52 (2.54) | <0.001 |
| qSOFA | 0.45 (0.59) | 1.0 (0.60) | 0.41 (0.57) | <0.001 |
| CRB-65 | 0.18 (0.47) | 0.61 (0.78) | 0.14 (0.42) | <0.001 |
| Kanagawa score | 9.29 (5.44) | 15.67 (3.31) | 8.80 (5.26) | <0.001 |
- Abbreviations: CRB-65, Confusion, Respiratory rate, Blood pressure, and Age ≥65; NEWS, National Early Warning Score; qSOFA, Quick Sequential Organ Failure Assessment; SD, standard deviation.
We found the following values to be considered as candidate cut-off points for each score: cut-off point of 7 for the NEWS, 1 for the qSOFA, 1 for the CRB-65, and 14 for the Kanagawa score (Table 3). Sensitivity analysis results with candidate cut point for each score are listed in Table 4.
| Points | NEWS | qSOFA | CRB-65 | Kanagawa score | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Sensitivity | Specificity | Youden's index | Sensitivity | Specificity | Youden's index | Sensitivity | Specificity | Youden's index | Sensitivity | Specificity | Youden's index | |
| 0 | 100 | 0 | 0 | 100 | 0 | 0 | 100 | 0 | 0 | 100 | 0 | 0 |
| 1 | 100 | 0 | 0 | 83.3 | 62.7 | 46.1 | 44.4 | 88.3 | 32.8 | 100 | 0 | 0 |
| 2 | 100 | 0 | 0 | 16.7 | 96.1 | 12.8 | 16.7 | 97.4 | 14.1 | 100 | 0 | 0 |
| 3 | 100 | 42.4 | 42.4 | 0 | 100 | 0 | 0 | 100 | 0 | 100 | 0 | 0 |
| 4 | 94.4 | 53.7 | 48.1 | 100 | 0 | 0 | ||||||
| 5 | 88.9 | 65.0 | 53.0 | 100 | 0 | 0 | ||||||
| 6 | 77.8 | 76.6 | 54.4 | 100 | 0 | 0 | ||||||
| 7 | 72.2 | 86.2 | 58.4 | 100 | 0 | 0 | ||||||
| 8 | 61.1 | 93.1 | 54.2 | 100 | 0 | 0 | ||||||
| 9 | 38.9 | 97.4 | 36.3 | 100 | 0 | 0 | ||||||
| 10 | 22.2 | 98.7 | 20.9 | 100 | 55.9 | 55.9 | ||||||
| 11 | 11.1 | 100 | 9.8 | 94.4 | 60.6 | 55.1 | ||||||
| 12 | 5.56 | 100 | 5.6 | 88.9 | 66.7 | 55.6 | ||||||
| 13 | 0 | 100 | 0 | 88.9 | 74.0 | 62.9 | ||||||
| 14 | 83.3 | 79.7 | 63 | |||||||||
| 15 | 50 | 84.0 | 34 | |||||||||
| 16 | 38.9 | 88.7 | 27.6 | |||||||||
| 17 | 33.3 | 92.6 | 26 | |||||||||
| 18 | 33.3 | 94.4 | 27.7 | |||||||||
| 19 | 22.2 | 96.5 | 18.8 | |||||||||
| 20 | 22.2 | 99.1 | 21.4 | |||||||||
| 21 | 5.6 | 99.6 | 5.1 | |||||||||
| 22 | 5.6 | 100 | 5.6 | |||||||||
| 23 | 0 | 100 | 0 | |||||||||
- Note: Underlined highlighted scores are recommended cut-off points based on Youden's index. Coordinate points of the receiver operating characteristic curves in Figure 1 are listed with the corresponding sensitivity and specificity.
- Abbreviations: COVID-19, coronavirus disease 2019; CRB-65, Confusion, Respiratory rate, Blood pressure, and Age ≥65; NEWS, National Early Warning Score; qSOFA, Quick Sequential Organ Failure Assessment.
| Sensitivity % (95% CI) | Specificity % (95% CI) | PPV % (95% CI) | NPV % (95% CI) | |
|---|---|---|---|---|
| The conventional cut-off points | ||||
| NEWS ≥7 | 72.2 (46.5–90.3) | 86.2 (81.0–90.3) | 36.7 (27.4–47.1) | 96.5 (93.0–98.3) |
| qSOFA ≥2 | 16.7 (3.6–41.4) | 96.1 (92.7–98.2) | 32.2 (12.4–61.6) | 91.2 (89.4–92.7) |
| CRB-65 ≥3 | 0 (0.0–18.5) | 100.0 (98.4–100.0) | NA | 90 (90.0–90.0) |
| Kanagawa score ≥5 | 100.0 (81.5–100.0) | 21.7 (16.5–27.5) | 12.4 (11.7–13.2) | 100.0 |
| The new proposed cut-off points | ||||
| NEWS ≥7 | 72.2 (46.5–90.3) | 86.2 (81.0–90.3) | 36.7 (27.4–47.1) | 96.5 (93.0–98.3) |
| qSOFA ≥1 | 83.3 (58.6–96.4) | 62.8 (56.2–69.0) | 19.9 (16.0–24.5) | 97.1 (92.3–99.0) |
| CRB-65 ≥1 | 44.4 (21.5–69.2) | 88.3 (83.5–92.2) | 29.7 (18.4–44.1) | 93.5 (90.4–95.6) |
| Kanagawa score ≥14 | 83.3 (58.6–96.4) | 79.7 (73.9–84.7) | 31.3 (24.7–38.7) | 97.7 (93.9–99.2) |
- Note: The table presents information on discrimination measures of clinical risk scores at admission to predict severe disease from the COVID-19 Delta variant for selected cut-off points.
- Abbreviations: CI, confidence interval; COVID-19, coronavirus disease 2019; CRB-65, Confusion, Respiratory rate, Blood pressure, and Age ≥65; NA, not applicable; NEWS, National Early Warning Score; NPV, Negative Predictive Value; PPV, Positive Predictive Value; qSOFA, Quick Sequential Organ Failure Assessment.
Figure 1 and Table 4 show the performance of the studied clinical risk scores to predict severe disease. The highest area under the curve (AUROC) was found for the NEWS. The NEWS predicted severe disease with the highest AUC (0.88, 95% CI 0.83–0.92). A NEWS ≥7 at admission predicted severe disease with 72.2% sensitivity and 86.2% specificity (Table 4). There was no significant difference between the NEWS and Kanagawa score (AUROC 0.86, 95% CI 0.81–0.90; DeLong's test p = 0.63). The NEWS was superior to the qSOFA (AUROC 0.74, 95% CI 0.69–0.80; DeLong's test p < 0.01) and CRB-65 (AUROC 0.67, 95% CI 0.61–0.73; DeLong's test p < 0.01; Appendix S3) scores.
DISCUSSION
In the pandemic situation, early identification of patients at risk of severe disease and judgment as to whether to be hospitalized is crucial. The NEWS is widely used in clinical practice. Furthermore, in the context of COVID-19, the NEWS has so far been evaluated,6-8 but the population of these studies were selected from March 2020 to April 2020 when the Delta variant did not exist. In this study, compared with COVID-19 wild type, the NEWS showed comparable prediction accuracy for severe disease caused by the Delta variant. The AUROC of the NEWS for suspected COVID-19 Delta variant infection is similar to that of the NEWS for COVID-19 wild type as reported by Covino et al (0.88 vs 0.82, respectively).6
There was a significant difference between the severe group and the nonsevere group in all four scores (NEWS, qSOFA, CRB-65, and Kanagawa score). We verified which of these four scores was the most accurate. A NEWS ≥7 at hospital admission appears superior to both qSOFA and CRB-65 in prediction of severe disease and in-hospital mortality from suspected COVID-19 Delta variant infection, and this is likely due to the allotment of points for respiratory rate, oxygen saturation, and pulse rate, which are the scores of three vital items with significant differences.
The Kanagawa score performs as well as the NEWS, with overlapping CIs and nonsignificant p value. The recommended cut-off value for the Kanagawa score in this study was 14 points, which was much higher than the conventional 5 points. The complexity of using the Kanagawa score makes the NEWS a more feasible option in clinical practice. Especially in the situations where there is no time to take history or computed tomography images, it is desirable to use the NEWS, which is calculated only from vital signs, for triage.
Larger studies are needed to confirm the results of our study, and to investigate the optimal cut-off value for clinical use.
Limitations
The current study has limitations. The small study size is the most prominent. The lack of dialysis patients limits our ability to generalize the results. Our results may be considered preliminary to evaluate the COVID-19 Delta variant.
CONCLUSIONS
A NEWS ≥7 at admission predicted severe disease in patients with suspected COVID-19 Delta variant infection and was superior to both qSOFA and CRB-65. The Kanagawa score appears to perform well, albeit it is more complex to administer. The use of clinical scoring systems to predict severe disease and mortality in patients with COVID-19 Delta variant infection should be validated further in larger prospective studies.
ACKNOWLEDGMENTS
The authors thank all COVID-19 medical teams at International University of Health and Welfare Narita Hospital.
FUNDING INFORMATION
This article was supported by grants from Marumo Emergency Medical Research Promotion Fund 2021.
CONFLICT OF INTEREST STATEMENT
The authors declare that they have no competing interests.
ETHICS STATEMENT
Approval of the Research Protocol with Approval No. and Committee Name: All study procedures involving human participants were conducted in accordance with the standards of the Institutional Review Board of the International University of Health and Welfare (approval number 20-Nr-054) and conformed to the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.
Informed Consent: The requirement for informed consent was waived by the ethics committee because this retrospective analysis was limited to preexisting data collected as part of the standard of care by physicians; data and privacy issues are protected.
Registry and the Registration No. of the Study/Trial: N/A.
Animal Studies (if applicable): N/A
Open Research
DATA AVAILABILITY STATEMENT
The data sets analyzed during this study are available from the corresponding author on reasonable request.
