Artificial neural network identifies nonsteroidal anti-inflammatory drugs exacerbated respiratory disease (N-ERD) cohort

1 INTRODUCTION

Nonsteroidal anti-inflammatory drug (NSAID)–exacerbated respiratory disease (N-ERD) is characterized by the presence of asthma, chronic rhinosinusitis with nasal polyposis (CRSwNP), and acute respiratory reactions precipitated by aspirin (acetylsalicylic acid) or other NSAIDs. Previous studies have demonstrated that patients with N-ERD experience a more severe course of asthma, require higher doses of corticosteroids to control the disease, and suffer from a particularly aggressive polypoid hypertrophy of the sinus mucosa compared with individuals without N-ERD.1, 2 The prevalence of N-ERD is about 7% in adult patients with asthma and twice as high in those with severe asthma.3

N-ERD is a heterogeneous disease, usually associated with eosinophilic inflammation in the airways, although neutrophilic, mixed, or paucigranulocytic cell phenotypes based on induced sputum (IS) can also be found in some patients.1, 4 Another key feature of N-ERD is the dysregulation of arachidonic acid metabolism pathways,1, 5 including a decrease in anti-inflammatory prostaglandin E₂ (PGE₂) levels and an overproduction of proinflammatory eicosanoids such as cysteinyl leukotrienes and prostaglandin D₂ (PGD₂). The imbalance of these lipid mediators has been observed in various body fluids, including urine and IS supernatant (ISS).4-6

Despite a relatively broad knowledge on the pathophysiology of N-ERD,1 only a few in vitro diagnostic tests have been proposed. The investigated parameters included basophil activation7; 15-hydroxyeicosatetraenoic acid generation in peripheral blood leukocytes8; plasma eosinophil–derived neurotoxin levels9; serum periostin levels10; serum leukotriene E₄ (LTE₄) levels and the ratio of LTE₄ to prostaglandin F_2α11; and urinary LTE₄ levels.12 Unfortunately, none of these parameters can be recommended for the purpose of diagnosing N-ERD. In clinical practice, the most reliable diagnostic method is an intranasal, bronchial, and oral aspirin provocation test, the latter being the gold standard.13-15 However, oral aspirin challenge is time consuming and carries a risk of severe hypersensitivity reactions. Moreover, bronchial and oral aspirin challenge tests are difficult to perform in individuals with uncontrolled asthma whose forced expiratory volume in the first second (FEV₁) is below 70% of the predicted value.14, 15 Thus, it can only be performed in a limited number of patients. Therefore, safe methods that could facilitate the diagnosis of N-ERD are needed. Cahill et al suggested that an informatics algorithm applied to electronic health records could help identify patients with N-ERD.16

The most common machine learning techniques are artificial neural networks (ANNs) and support vector machines (SVMs). The ANN is a type of computer software inspired by biological neurons. The main concept of the ANN is a learning ability to recognize complex patterns on the basis of imputed information (variables and outcomes), which can then be used for solving similar problems in the future.17 Machine learning techniques are gaining popularity in the field of medicine18-20 because they are noninvasive, easy to implement, and cost-effective. The aim of the present study was to design the first artificially intelligent program based on clinical characteristics, sputum cell phenotypes, and eicosanoid levels in ISS and urine, which could predict the diagnosis of N-ERD. This approach is not burdensome for patients, and sputum induction is now almost a routine procedure in specialized clinical centers, where treatment of asthma is tailored to individual patient needs based on sputum cell phenotypes.

2 MATERIAL AND METHODS

2.5 Artificial neural network

An ANN is based on input variables (input layer), which are integrated in a nonlinear process (hidden layer) to finally generate output results (output layer). The ANN model is initially fitted on a training sample, which comprises data records used to train the neural network. An independent set of data named the testing sample (or holdout sample) is used to set the architecture of the ANN and prevent overtraining. Finally, another set of data records, called the validating sample, is used to assess the final neural network. The errors of the validating sample give an “honest” estimate of the predictive ability of the ANN because the validating cases are not used to build the model.17

We used 143 cases (70.4%) of data (87 N-ERD and 56 ATA cases) as the training sample, among which 23 cases were separated as the testing sample. Then, 60 cases (29.6%) of data (34 N-ERD and 26 ATA cases) were used as the validation sample. We included 18 variables (14 qualitative and 4 quantitative; Table 2) in the input layer, which consisted of 36 neurons. We attempted to determine the best performance using different (1–5) numbers of ANN hidden layers and found that the best performance in our ANN was achieved with a single hidden layer constructed with eight neurons (Figure 1). The ANN was trained using a backpropagation algorithm. The input quantitative variables were previously logarithmically normalized. The activation function was logistic in the hidden layer and linear in the output layer. The higher output number of our ANN indicated a higher probability of N-ERD. In order to convert the fractional N-ERD prediction value into a binary N-ERD status (yes or no), the algorithm introduced a cutoff value above which our ANN would predict a “yes” N-ERD status. Once the training was complete, the algorithm tested a number of cutoff values to establish the sensitivity and specificity for the prediction of N-ERD. When the cutoff value was selected from the training data set, the same value was then used on the validation set, and, similarly, the sensitivity and specificity were computed.

3 RESULTS

3.1 Artificial neural network

The training sensitivity (with 95% CI) of the ANN for the prediction of N-ERD was 95.40% (88.64%-98.73%); specificity, 87.50% (75.93%-94.82%); and accuracy, 92.31% (86.65%-96.10%). The validation sensitivity was comparable and reached 94.12% (80.32%-99.28%); specificity, 73.08% (52.21%-88.43%); and accuracy, 85.00% (77.43%-92.90%). The AUROC was 0.85 (0.73-0.93) and 0.83 (0.71-0.90) for training and validation, respectively (Figures 2 and 3). The diagnostic odds ratio (with 95% CI) was 145.25 (40.46-278.3) and 43.43 (8.17-521.48) for training and validation, respectively (Table 3).

Table 3. Comparison of the 3 models for determining nonsteroidal anti-inflammatory drug–exacerbated respiratory disease among patients with asthma: artificial neural network (ANN), support vector machine (SVM), and multiple logistic regression (MLR) in the validation cohort

	AUROC (95% CI)	P	Sensitivity (%)	Specificity (%)	DOR (95% CI)
ANN	0.83 (0.71-0.91)	<.001	94.12	73.08	43.43 (8.17-521.49)
SVM	0.75 (0.62-0.85)	<.001	82.35	61.54	7.47 (4.08-13.66)
MLR	0.79 (0.67-0.88)	<.001	82.35	69.23	6.14 (1.98-19.04)

• Abbreviations: AUROC, area under the receiver operating characteristic curve; CI, confidence interval; DOR, diagnostic odds ratio.

4 DISCUSSION

There are no reliable in vitro tests that would allow to diagnose N-ERD with high probability. None of the single biomarkers have been generally recognized as being able to determine the presence of the disease in clinical practice. Our center has been involved in research on N-ERD for many years. It was initiated by Professor Andrzej Szczeklik, who presented the cyclooxygenase theory of aspirin-induced asthma, which indicated that aspirin hypersensitivity is related to the pharmacological inhibition of cyclooxygenase.26 We have been investigating various diagnostic methods for aspirin hypersensitivity, including in vitro tests, yet without much success. In a recent systematic review and meta-analysis, Hagan et al have shown for the first time that the measurement of urinary LTE₄ could be used as a clinical test to identify the risk of aspirin intolerance among patients with asthma.27 However, further research is needed to standardize the normative values for each of the major methodologies used to analyze urinary LTE₄ levels and define the clinical utility of this biomarker. So far, the prediction of N-ERD diagnosis can only be established on the basis of a patient's medical history and a positive aspirin challenge test.

As a medical and research center dealing with N-ERD patients, we have hundreds of individuals with N-ERD and ATA in our cohort. On various occasions, including research grants, our patients have been subjected to aspirin challenge test as well as clinical and biochemical evaluations. These included the assessment of asthma severity and control, spirometry, computed tomography scans of the sinuses, total serum IgE levels, skin tests, cell phenotypes based on sputum, as well as eicosanoid evaluation in ISS and urine. Using this prospectively collected data, we aimed to develop an artificially intelligent system for predicting the presence of N-ERD among asthma patients. Artificial intelligence is the ability of a learning machine to interpret external data, learn from it, and use those results to achieve certain goals. The ANN is an artificial intelligence tool that has recently become popular in medical diagnosing.17, 18 It is a human brain–inspired computing system, which is able to process complex data (input variables and known outcomes), and then use the recognized patterns to solve similar problems in the future. In the present study, we have for the first time developed an ANN that transformed a combination of 18 clinical and biochemical variables of asthma patients into an output value differentiating between N-ERD and ATA.

The selected clinical and biochemical data are easily obtainable in a clinical setting and seem to be helpful in predicting the presence of N-ERD. Six weeks prior to the study, all participants were treated in the same way with inhaled corticosteroids, long-acting beta₂-agonists, nasal corticosteroids, and, in 15 cases, with small doses of oral corticosteroids. The clinical differences between the 2 groups in our study concerned sex, the age of asthma onset, baseline FEV₁, treatment, the presence of CRSwNP, and a history of sinonasal surgery. Such discrepancies between N-ERD and ATA are well recognized and were previously acknowledged by other authors.1 Apart from blood and urine collection, all study participants underwent sputum induction, as it is a relatively noninvasive procedure that allows the parallel assessment of inflammation cellular pattern and eicosanoid profile in the bronchi. Assessment of cell phenotypes based on sputum is currently a routine procedure in clinical centers providing care for asthma patients. However, ISS biomarkers are only evaluated in specialist laboratory centers. Therefore, we attempted to exclude the phenotype based on IS, ISS eicosanoids (still including cell phenotypes based on IS), and then all IS variables (excluding cell phenotypes and ISS eicosanoids) from our machine learning model, and then performed 3 additional analyses based on the remaining parameters. Our results showed that the separate exclusion of the phenotype based on IS or ISS eicosanoids and the development of ANN based on the remaining parameters deteriorated the specificity of the ANN from 73% to 57.7% or 61.5%, respectively. Additionally, we performed an analysis excluding solely urinary LTE₄, as the overproduction of cysteinyl leukotrienes is the key pathologic mechanism in N-ERD. Our results showed that the exclusion of this single parameter deteriorated the specificity of the ANN from 73% to 65.4%. This indicates that both the phenotype based on IS as well as ISS and urinary eicosanoid concentrations are important in the prediction of N-ERD diagnosis. The ANN developed with the exclusion of even one of the above parameters was characterized by markedly lower specificity. It could also be expected that sputum induction would gain popularity in the era of precision medicine.28

N-ERD is usually associated with eosinophilic inflammation of the airways, but other sputum phenotypes (ie, neutrophilic, mixed, or paucigranulocytic) may also occur.4 Another key feature of N-ERD is the complex alteration of arachidonic acid metabolism pathways.1, 5 The imbalance between pro- and anti-inflammatory lipid mediators was previously observed in the ISS and urine of N-ERD individuals,4-6 and this determined the selection of PGD₂, PGE₂, and LTE₄ for our machine learning analysis. We observed significantly higher baseline levels of PGD₂ and LTE₄ in ISS as well as urinary LTE₄ in N-ERD individuals. On the contrary, baseline ISS PGE₂ levels did not differ between the two groups, which had also been reported previously.5 The inclusion of this lipid mediator in our analysis was prompted by the fact that PGE₂ is one of the key eicosanoids involved in the pathogenesis of N-ERD.1 Moreover, the significant decrease of ISS PGE₂ concentrations in patients with N-ERD occurs during oral aspirin challenge.21

The present study has shown for the first time that the ANN could be successfully used to differentiate between patients with N-ERD and ATA. A multi-parameterized ANN was characterized by the highest sensitivity, specificity, and accuracy for the prediction of N-ERD among the 3 investigated methods including also the SVM and MLR. The ANN and SVM concentrate on finding a useful computer algorithm for solving discrimination problem and can be treated as competing solutions. The MLR is a model-based technique, in which two or more measurement variables are taken into consideration simultaneously to predict a value of one nominal variable.25 In our study, the MLR yielded the worst results among the investigated methods and should not be recommended for N-ERD prediction based on the analyzed variables. The sensitivity of our ANN for N-ERD prediction was 94%, which is a good result when compared with other studies using artificial intelligence to predict the diagnosis of various diseases.18, 29 In comparison with the oral aspirin challenge test,14 our ANN was characterized by higher sensitivity (94% vs 89%) but lower specificity (73% vs 93%) for N-ERD prediction. Similarly, in comparison with bronchial aspirin challenge test,14 our ANN was characterized by higher sensitivity (94% vs 77%) and lower specificity (73% vs 93%) for N-ERD prediction. The above provocation methods were perfected many years ago. For example, the oral aspirin test would take 2 days, while currently, a shorter 1-day oral challenge test is used. It is not known how our results would compare to the 1-day aspirin challenge protocols.15 We believe that it would also be very interesting and useful to perform a similar machine learning study between N-ERD and ATA only with CRSwNP. In our study, the number of ATA with CRSwNP patients was too small to perform a reliable analysis.

Our approach obviously cannot replace the gold-standard diagnostic method, which is the oral aspirin challenge test. However, the implementation of the ANN might provide an added value for identifying patients with N-ERD in clinical practice (Figure 4). This method is time effective, and, unlike the aspirin challenge test, it does not require patient hospitalization. All necessary procedures and data collection can be conducted in an outpatient setting during a single day. The ANN could also be helpful in predicting N-ERD in patients with asthma and low baseline FEV₁. The inclusion of patients with low FEV₁ in our study was only limited by sputum induction, for which the minimum FEV₁ value is 50%.30 The oral aspirin challenge test can be carried out when baseline FEV₁ is at least 70%.14, 15 In contrast to the aspirin challenge test, the procedures necessary to obtain data for the presented ANN do not carry a risk of life-threatening reactions. Recently, it has been reported that some asthma patients with aspirin hypersensitivity have negative aspirin challenge test results when they receive leukotriene receptor antagonists31 or had undergone sinus surgery shortly before aspirin provocation.32 However, there were no such participants in our study, so it is difficult to conclude whether the ANN would be also useful in situations where the aspirin challenge is negative in the face of a strong clinical suspicion.

The results of this study should be interpreted within the context of its limitations. First, the clinical and biochemical assessment was performed directly before the aspirin challenge test in all patients with N-ERD and ATA. Therefore, asthma treatment was modified.14 Second, it may seem that the number of 18 variables included in our ANN is high, but the exclusion of IS variables substantially reduced the percentage of correctly classified diagnoses. Third, the group of 203 patients could seem to be relatively small for the development of a machine learning system. However, it should be emphasized that N-ERD is a specific endotype of asthma and the number of patients is limited. We hope to collaborate on the development of the ANN with our colleagues from other centers in the future. Finally, as for now, our ANN is only a statistical model, which in order to be used in clinical practice will require the development of a user interface available in the physician's office.

In conclusion, we have developed a multi-parameterized ANN that is able to identify patients with N-ERD among the asthmatic population. Although it cannot be perceived as a replacement for the aspirin challenge test, it shows that machine learning techniques might be beneficial in the diagnostic process of N-ERD. Further research is needed to develop a system that would allow for an entirely in vitro diagnosis of N-ERD via the implementation of machine learning techniques.

CONFLICT OF INTEREST

None of the authors have a conflict of interest in relation to this work.

AUTHOR'S CONTRIBUTIONS

KET and LM involved in project concept, study design, and study implementation; contributed to writing of the first draft of the manuscript. KET, AC, and LM involved in data and statistical analysis. All authors contributed to data collection and manuscript editing; reviewed and approved the final version of the manuscript.