Volume 6, Issue 1 e962
NARRATIVE REVIEW
Open Access

Application of machine learning methods in predicting schizophrenia and bipolar disorders: A systematic review

Mahdieh Montazeri

Mahdieh Montazeri

Department of Health Information Sciences, Faculty of Management and Medical Information Sciences, Kerman University of Medical Sciences, Kerman, Iran

Medical Informatics Research Center, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran

Contribution: Project administration

Search for more papers by this author
Mitra Montazeri

Corresponding Author

Mitra Montazeri

Medical Informatics Research Center, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran

Correspondence Mitra Montazeri, Medical Informatics Research Center, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Haft-bagh highway, PO Box: 7616911320, Kerman, Iran.

Email: [email protected] and [email protected]

Contribution: Formal analysis, Writing - review & editing

Search for more papers by this author
Kambiz Bahaadinbeigy

Kambiz Bahaadinbeigy

Medical Informatics Research Center, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran

Contribution: Conceptualization

Search for more papers by this author
Mohadeseh Montazeri

Mohadeseh Montazeri

Department of Computer, Faculty of Fatimah, Kerman Branch Technical and Vocational University, Kerman, Iran

Contribution: Data curation, Writing - original draft

Search for more papers by this author
Ali Afraz

Ali Afraz

Medical Informatics Research Center, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran

Contribution: Conceptualization, Writing - original draft

Search for more papers by this author
First published: 28 December 2022
Citations: 1

Abstract

Background and Aim

Schizophrenia and bipolar disorder (BD) are critical and high-risk inherited mental disorders with debilitating symptoms. Worldwide, 3% of the population suffers from these disorders. The mortality rate of these patients is higher compared to other people. Current procedures cannot effectively diagnose these disorders because it takes an average of 10 years from the onset of the first symptoms to the definitive diagnosis of the disease. Machine learning (ML) techniques are used to meet this need. This study aimed to summarize information on the use of ML techniques for predicting schizophrenia and BD to help early and timely diagnosis of the disease.

Methods

A systematic literature search included articles published until January 19, 2020 in 3 databases. Two reviewers independently assessed original papers to determine eligibility for inclusion in this review. PRISMA guidelines were followed to conduct the study, and the Prediction Model Risk of Bias Assessment Tool (PROBAST) to assess included papers.

Results

In this review, 1243 papers were retrieved through database searches, of which 15 papers were included based on full-text assessment. ML techniques were used to predict schizophrenia and BDs. The main algorithms applied were support vector machine (SVM) (10 studies), random forests (RF) (5 studies), and gradient boosting (GB) (3 studies). Input and output characteristics were very diverse and have been kept to enable future research. RFs algorithms demonstrated significantly higher accuracy and sensitivity than SVM and GB. GB demonstrated significantly higher specificity than SVM and RF. We found no significant difference between RF and SVM in terms of specificity.

Conclusion

ML can precisely predict results and assist in making clinical decisions-concerning schizophrenia and BD. RF often performed better than other algorithms in supervised learning tasks. This study identified gaps in the literature and opportunities for future psychological ML research.

1 INTRODUCTION

Schizophrenia and bipolar disorder (BD) are critical and high-risk inherited mental disorders that have debilitating symptoms.1 These disorders are among the severe psychiatric diseases that have many overlaps and similarities with each other and affect the patient's behavior in the family and society. According to the World Health Organization, these disorders are among the top 10 causes of disability worldwide.2 Schizophrenia and BD affect 3% of the world's population.3, 4 Patients with schizophrenia and BD have a higher mortality rate than the general population.5 One of the prominent causes of death in these patients is suicide. In Danish registers, the rate of suicide is reported as 7.8% in men and 4.9% in women with BD.6

There are Five key features that define schizophrenia and BD and other psychotic disorders. These include delusions, hallucinations, disorganized thinking (inferred from speech), grossly disorganized or abnormal motor behavior, and negative symptoms.7 As compared with other disorders, schizophrenia and BD specifically, the presence or absence of specific psychotic symptoms identified as first-rank symptoms (auditory hallucinations; thought withdrawal, insertion, or interruption; thought broadcasting; somatic hallucinations; delusional perception; feelings or actions controlled by external agents) may be particularly helpful for making the diagnosis.8-10

Many patients with schizophrenia and BD experience a long clinical period.7 Symptoms of the disease begin between the ages of 16 and 30. These symptoms fall into three categories: positive (hallucinations, delusions, and mental disorders), negative (lack or absence of facial expressions, feelings of little pleasure, and decreased sense of speech), and cognitive (concentrating and maintaining difficulty).8-10 BD patients experience persistent changes in brain structures, such as enlargement of the third and lateral ventricles of the brain and a decrease in the volume of gray matter in the anterior and middle cerebral cortex, cortical and mesotemporal cortex, and decrease in the posterior abdominal callus volume.11 The economic costs associated with the disease vary from $94 million to $102 billion each year.12 Therefore, this disease imposes a heavy financial burden on the patients, their families, and society.13 Predicting the disease can go a long way in preventing it and controlling its costs. Since it takes an average of 10 years from the onset of the first symptoms to the definitive diagnosis of the disease,14 current approaches cannot effectively diagnose these diseases. Machine learning (ML) techniques are proposed as an effective tool to meet this need.

ML is a domain of artificial intelligence that allows computer algorithms to learn patterns by studying data directly without being explicitly programmed.15 Artificial intelligence using ML is entering the realm of medicine at an increasing pace and has been tested in various clinical applications ranging from diagnosis to outcome prediction.16 The utilization of ML techniques has many advantages, such as recognizing diseases, reducing physician decision-making errors, reducing healthcare costs, and improving the performance of healthcare providers.17

Various models have contributed significantly to the health domain, from rule-based systems to advanced ML models (deep learning). These models have been used in prediction, diagnosis, and treatment in healthcare, such as predicting survival in breast cancer,18 diagnosis and prognosis of COVID-19,19, 20 level of lung cancer,21 etc. ML techniques are also used to diagnose, classify, and predict schizophrenia and BD.22-26 Several ML methods have been used to predict the negative symptoms of schizophrenia based on speech signals22, 27 and to predict the recurrence of schizophrenia.24 Also, many studies have been conducted on the extraction of various features of computed tomography scans and magnetic resonance imaging (MRI) images in the diagnosis and prevention of schizophrenia and BD.28-30

Several algorithms, such as random forests (RF), support vector machine (SVM), and gradient boosting (GB), have been frequently used in this area. The RF method is fast, adaptable, and reliable for mining high-dimensional data. As the name suggests, an ensemble of many decision trees makes up a RF. The RF produces a classification for each tree, and the class voted on the most becomes the prediction.31 SVMs are linear models for classification and regression problems. Several practical problems can be solved through this technique, including linear and nonlinear problems. This algorithm generates a line or a hyperplane to classify the data into classes.32 A GB algorithm trains many models (typically decision trees) in sequential and additive order. The purpose of boosting is to transform weak classifiers into strong classifiers. Each new model in GB is designed to minimize prediction error as much as possible.33

This study aimed to conduct a systematic review of ML algorithms for predicting schizophrenia and BD to help early and timely diagnosis of the diseases to improve patients’ health.

2 MATERIALS AND METHODS

2.1 Information source and search

A systematic search was conducted in PubMed, Web of Science, and Scopus for relevant studies published before January 18, 2020. PRISMA guidelines were followed to conduct this study.34 Two groups of keywords related to: (A) ML and (B) schizophrenia and BD were used to search these databases. The keywords used to identify relevant papers are shown in Appendix 1.

2.2 Inclusion and exclusion criteria

All studies applying ML techniques for predicting schizophrenia and BDs were considered. We included original studies. The search was restricted to English-language publications. Editorials, commentaries, letters, books, presentations, and conference papers were excluded. All types of review studies were also excluded to prevent duplication in data collection.

2.3 Study selection

The selection process was initiated by removing duplicated papers. Then, two authors (MM, MM) independently reviewed the titles and abstracts of all identified studies. The same authors independently reviewed the relevant papers (MM, MM). The disagreements were resolved through discussion and, if required, referred to a third researcher (KB). The reasons for the exclusion of each study were documented during the screening process of the papers. Rayyan QCRI systematic review, a free web and mobile application platform, was used for paper screening.35

We additionally evaluated reference lists of relevant papers for relevant publications.

2.4 Data extraction and synthesis

We developed an Excel data-extraction form to extract specific details of each paper (Appendix 2). Two reviewers (MM, MM) Completed the form. This form consisted of study's location, data utilized, sample size, ML model, accuracy, sensitivity, specificity, area under the receiver operating characteristic curves (AUC), and precision (Table 1). A more detailed table of the PROBAST results is shown in Appendix 3.

Table 1. Characteristics of included studies
Disorder Data utilized What is predicted? Sample size Machine learning model Accuracy (%) Sensitivity (%) Specificity (%) AUC (%) Precision (%) Risk of bias First author Year
Schizophrenia EEG Having Schizophrenia 30 SVM 78.05 NR NR NR NR High Taylor et al.37 2017
GPC 80.49 NR NR 87 NR
Schizophrenia sMRI, fMRI Cognitive functions 324 Meta-analytic cognitive priors 70.9 (For mental domains) NR NR NR NR High M. Karrer et al.43 2019
70.8 (For experimental tasks)
Schizophrenia Symptomatic and MRI Symptom severity 167 LR NR NR NR 81 NR Low Talpalaru et al.44 2019
SVM NR NR NR 78 NR
RF NR NR NR 75 NR
Schizophrenia MRI Neurotic, and psychotic symptoms 34 SVM 94 76 100 NR NR Low Zarogianni et al.38 2016
Schizophrenia Biobank Having Schizophrenia 1606: RBF NR NR NR NR NR High Bracher-Smith et al.41 2019
803 SCZ SVM NR NR NR NR NR
803 HC RF NR NR NR NR NR
GBM NR NR NR NR NR
NN NR NR NR NR NR
LR NR NR NR NR NR
Schizophrenia Data Psychotic relapse 864: CART 63.8 71.0 44.8 NR NR Low Fond et al.24 2019
549 baseline
315 2 years
Schizophrenia Whole exome sequencing of genotypes and phenotypes Having Schizophrenia 5090: XGBoost 85.7 84.9 86.6 NR 86.9 Low Trakadis et al.42 2018
2545 SCZ L1.Logistic 74.6 72.0 77.3 NR 76.0
2545 HC SVM 70.7 70.8 70.6 NR 70.5
RF 81.7 82.0 81.3 NR 81.1
Schizophrenia MRI, fMRI Having Schizophrenia 211: Ridge 87 NR NR NR NR Unclear Salvador et al.29 2019
96 SCZ Lasso 80 NR NR NR NR
115 HC RF NR NR NR NR NR
GB NR NR NR NR NR
Schizophrenia fMRI Having Schizophrenia 86 SVM 94.12 1 89.47 94.73 NR Low Nimkar et al.26 2018
MRI C5.0 DT 91.18 1 84.21 91 NR
RF 91.18 1 84.21 92.1 NR
K-NN 79.41 1 63.16 81.579 NR
LDA 79.41 80 78.95 79.47 NR
GP 92 1 864 92.4 NR
NB 85.29 86.67 84.21 85.43 NR
Schizophrenia fMRI High-symptomatic patients vs. non/low-symptomatic 174: EM 86.9 79.8 93.1 NR 91.9 Low Kalmady et al.47 2019
81 SCZ
93 HC
Schizophrenia sMRI Having Schizophrenia 606: SVM 69 69 68 74 NR Low de Pierrefeu et al.40 2018
276 SCZ Enet 71 73 68 76
330 HC Enet-TV 68 68 68 74
Schizophrenia fMRI Having Schizophrenia 144: NR NR NR NR NR NR High Silva et al.74 2014
sMRI 69 SCZ
75 HC
Bipolar MRI Symptom severity 94: SVR NR NR NR NR NR High Sartori et al.30 2018
35 BD
59 HC
Schizophrenia or bipolar disorder MRI Predict risk of developing Schizophrenia or bipolar disorder in offsprings 185: SVM 77 NR NR NR NR High Hillegers et al.39 2018
50 SCZ
82 BD
53 HC
Schizophrenia Bipolar MRI Having Schizophrenia 606 SVM 69 69 68 74 NR Low Pierrefeu et al.46 2018
ElasticNet 71 73 68 76 NR
GraphNet 70 69 71 75 NR
TV-Enet 68 68 68 74 NR
  • Note: “+” indicates high risk of bias/low concern regarding applicability; “” indicates low risk of bias/high concern regarding applicability; “?” indicates unclear risk of bias/unclear concern regarding applicability; NR = not reported.
  • Abbreviations: AUC, area under the receiver operating characteristic curves; BD, bipolar disorder; CART, classification and regression tree; C5.0 DT, C5.0 Decision Tree; EEG, electroencephalography; EM, ensemble model; EN, elastic net; Enet, elasticnet-total variation; Enet-TV, elasticnet-total variation; GB, gradient boosting; GBM, gradient boosting machines; GP, Gaussian process; GPC, Gaussian processes classifiers; HC, healthy controls; K-NN, K-nearest neighbor; LASS, least absolute shrinkage and selection operator; LDA, linear discriminant analysis; LR, multivariate logistic regression; L1.Logistic, Lasso regularized (L1) logistic regression; MRI, magnetic resonance imaging; NB, Naïve Bayes; NN, neural networks; ONR, one nonlinear regression; RBF, linear and radial basis function; RF, random forests; SCZ, schizophrenia; SVM, support vector machine; SVR, support vector regression; Xgboost, extreme gradient boosting.

2.5 Risk of bias (ROB) assessment

To assess the ROB, we used the Prediction Model Risk of Bias Assessment Tool (PROBAST).36 It is a tool for assessing the ROB and the applicability of diagnostic and prognostic prediction model studies. It includes 20 signaling questions across 4 domains (participants, predictors, outcome, and analysis). This explanation and elaboration document describes the rationale for including each domain and signaling question and guides researchers, reviewers, readers, and guideline developers to use them to assess the ROB and applicability concerns.

3 RESULTS

We retrieved 1243 papers through database searches. After title and abstract screening 144 papers were identified for full-text assessment. Full-text assessment excluded 129 studies due to various reasons. Fifteen papers met the inclusion criteria (Figure 1). An analysis of the algorithms applied, the inputs they were trained on, the outputs they were trained to predict, and their relative performance statistics are presented. An average number of 185 patients were used in each study (mean = 681.4, SD = 1289.65). The median number of ML algorithms employed in each study was two (mean = 2.67, SD = 1.99). Most of the studies (n = 13) applied ML algorithms only to schizophrenia disorder, and one study applied ML algorithms to both schizophrenia and BDs. All the included studies applied ML algorithms to predict the symptoms of these disorders. Data utilizes in prediction of 10 studies was based on MRI. In eight studies, ML algorithms predicted schizophrenia disorder while in three studies these algorithms predicted symptoms severity.

Details are in the caption following the image
Study identification and selection process

3.1 Publications and algorithms applied in schizophrenia and BD over time

Over the past decade, many publications applying ML to schizophrenia and BD decision support have increased rapidly. The top two most frequently applied algorithms were SVM and RF. As shown in Table 1, most algorithms have been recently applied to schizophrenia and bipolar.

3.2 Summary of literature included, and algorithms applied

Most studies reported outcomes for more than one ML technique.12, 38, 45 The most frequently developed models used the variants of ML models, including SVM in 10 studies in references,24, 30, 37-42, 44, 46 RF in 5 studies,24, 29, 41, 42, 44 GB in 3 studies29, 41, 42 with the  three different types of gradient boosting (GB) machines, Xgboost and GB, logistic regression in 3 studies41, 42, 44 and its extended L1.Logistic, decision tree in 2 studies24, 26 with two types of CART and C5.0 DT, and Gaussian process in 2 studies.26, 37 Several ML techniques each appeared in the one of the studies: Meta-analytic cognitive prior,43 ElasticNet,41 GraphNet,41 TV-Enet,41 linear and radial basis function,42 neural network,42 Ridge,29 Lasso,29 K-NN,26 linear discriminant analysis,26 Naïve Bayes,26 and ensemble model.47

3.3 Predictive model performance evaluation statistics

The performances of the models were evaluated in varied ways. Accuracy and sensitivity (recall)-specificity were the most frequently reported performance statistics (in 10 references24, 26, 29, 37, 39, 40, 42, 43, 46, 47 and 7 of studies,24, 26, 38, 40, 42, 46, 47 respectively). Five of the studies26, 37, 40, 44, 46 reported AUC, while Precision was reported by 2 studies.42, 47

3.4 Comparing algorithm performance

A comparison of the performance of the top three most frequently applied algorithms (SVM, RF, and GB) is shown in Table 2. Insufficient performance data were reported in this corpus to include other algorithms. We compared algorithms based on their accuracy, sensitivity, specificity, AUC, and Precision (Table 2). Multivariate logistic regression, Neural Network, and SVM differed significantly in their sensitivity performance. Based on this table, RF was significantly more accurate than SVM. The accuracy performance of RF and GB was not significantly different. The Specificity of the three algorithms was not significantly different. It showed that RF demonstrated significantly higher sensitivity than SVM.

Table 2. Statistical comparisons of reported performance metrics for support vector machine, random forests, and gradient boosting
Performance metrics, mean (SD; n)
Accuracy (%) Sensitivity (%) Specificity (%) AUC (%) Precision (%)
Support vector machine (SVM) 80.48 (11.08, 6) 78.95 (14.34, 4) 82.02 (15.34, 4) 82.24 (11, 3) 70.5 (–, 1)
Random forests (RF) 86.44 (6.7, 2) 91 (12.73, 2) 82.75 (2.06, 2) 83.55 (12.09, 2) 81.1 (–, 1)
Gradient boosting (GB) 85.7 (–, 1) 84.9 (–, 1) 86.6 (–, 1) 86.9 (–, 1)
  • Abbreviation: AUC, area under the receiver operating characteristic curves.

3.5 Classification performance

The most commonly reported performance metrics for schizophrenia and BD were sensitivity (recall), specificity, and accuracy.48, 49 Multiple studies also presented positive predictive value (precision)42, 47 and error matrix outcomes as the AUC.26, 37, 40, 44, 46 However, there was no overall consistency as to which specific measures were reported.

3.6 Predictive model validation

In this study, 10 of the 13 studies provided details of a validation process for the applied models.24, 26, 29, 30, 37-43, 46, 47 Different forms of internal cross-validation and holdout datasets were most commonly applied. K-fold cross-validation and leave-one-out cross-validation frameworks were used to split data into training, validation, testing sets, or optimizing model parameters.50

3.7 Risk of bias

We critically reviewed studies for ROB using PROBAST.36 Our analysis revealed that except for six, all studies had some bias due to a low number of participants, lack of external validation, and failure to meet the study's goal (Table 1).

4 DISCUSSION

In this study, the applications of ML to support clinical decision-making in schizophrenia and BD were reviewed. Results suggested that the use of ML in schizophrenia and BD is rapidly growing.51, 52 There is substantial room for further applications of ML technologies to schizophrenia and BD data. Many ML applications and modeling methods have been used based on the findings. A wide variety of successfully predicted outcomes could facilitate decision-making. The number of ML publications on schizophrenia and BD has rapidly increased over the past few years.53-55

According to this study, most schizophrenia and BD ML publications were in the domain of MRI and fMRI data. Few ML studies have focused on EEG data. The applied modeling approaches were substantially heterogeneous. Variation in the studies was related to algorithms applied, input and output variables used, and methods for assessing predictive model performance.

In this paper, SVM and RF were the most commonly applied algorithms, compared to other studies, applying SVMS56-59 and RF.60, 61 Consistent with a previous study,62 RF frequently outperformed most other algorithms on supervised learning tasks. Since both SVM and RF are discriminative, they can handle large amounts of data, and capture nonlinear relationships across input features.44 They were selected to predict disease outcomes, often demographics, clinical history, and investigation-related features were used.49 AUC, sensitivity, accuracy, specificity, and precision performance metrics were the most commonly reported.49 The accuracy of RF was significantly higher than that of SVM, and compared to SVM, the precision of GB was significantly higher. To facilitate future modeling, the input and output variables were synthesized. Gaps in knowledge and opportunities concerning the development of additional clinical decision support systems to improve the care of schizophrenia and BD patients were highlighted to clinicians and data scientists. To our knowledge, this systematic review is the first attempt to characterize the deployment of ML methods in schizophrenia and BD. We identified two different topic clusters, and keywords for each of the schizophrenia and BD subdomains by applying ML technologies. The use of these topics and keywords in the systematic review helped to deeply understand the past foci of the relevant literature and determine the current state of the knowledge. This understanding helps to identify gaps to be filled out by future researches. ML provides a series of technologies, from linear regression to RF, that can effectively predict outcomes for improving clinical decision-making in schizophrenia and BD. Based on the results, wherever large, labeled datasets were available, RF has been the top-performing algorithm in the ML methods due to its higher accuracy than multivariate logistic regression.62

Since RF performed better than other models, it can be proposed for predicting schizophrenia and BD. It is fast to apply63 and suitable for feature selection (finding efficient risk factors) alone.64 RF does not need overtraining65 and can handle data without preprocessing, e.g., do not need rescaling, transforming, or modifying data.65 The following items are the notable performances of RF:
  • Natural handling of “mixed” type data.65

  • Handling of missing values.66

  • Robustness to outliers in input space.66

  • Insensitive to the monotone transformation of inputs.67

  • Computational scalability.68

  • Ability to deal with irrelevant inputs.69

Comparing RF with SVM and GB, it is true that they have about the same accuracy. However, RF is more interpretable due to these facts:
  • Estimating feature importance during training for little additional computation.

  • Plotting of sample proximities.70

  • Visualizing output decision trees.70

RF is faster to train and can readily handle larger numbers of predictors. It employs fewer parameters71 and does not need cross-validation (it generates an internal unbiased estimate of the generalization error (test error) as the forest building progresses).

5 LIMITATIONS AND FUTURE RESEARCH

It is more likely to publish positive and significant findings.72, 73 Many of the included studies evaluated the performance of multiple algorithms and reported the inferior performance of some. However, since we could not find studies reporting failed ML modeling activities, publication bias and selective outcome reporting may have influenced our results. Few studies have comprehensively reported predictive model performance statistics. Hence, we had relatively small sample sizes for our statistical analysis of algorithm performance. Larger sample sizes could have demonstrated additional significant performance differences between algorithms. More research is suggested to examine AUC, accuracy, sensitivity, and specificity to facilitate robust model performance evaluation and compare studies. Future studies can contribute to the convergence of ML with schizophrenia and BDs. Future studies can use similar predictors and outcomes on bigger databases containing the information of patients with schizophrenia and BDs. This may help predict the same or additional outcomes that can be used to deploy systems based on these models. The number of software to predict and support schizophrenia and BDs decision-making is low. Further models have required these two disorders. Despite the richness of electronic health record datasets, they are not utilized sufficiently and are suitable for conducting ML studies. SVMs and transfer learning can utilize video data during real-time patient examination, diagnosis and prognosis, or intraoperative anatomical identification. SVMs can predict the postoperative status of patients on an hourly basis to provide safer proactive management of patient status.

6 CONCLUSION

ML effectively supports making clinical decisions before, during, and after the onset of schizophrenia and BDs. Due to the evident heterogeneity of apply ML in psychology and the popularity of distinctive ML studies, there is a substantial potential for future similar studies. Because of the accurate prediction of various operative outcomes, RF seems more effective than other algorithms in schizophrenia and BDs. Deploying ML by designing sound clinical decision support systems can lower complications and increase the quality and safety of health services provided to schizophrenia and BD patients.

AUTHOR CONTRIBUTIONS

Mahdieh Montazeri: project administration. Mitra Montazeri: formal analysis; writing—review & editing. Kambiz Bahaadinbeigy: conceptualization. Mohadeseh Montazeri: data curation; writing—original draft. Ali Afraz: conceptualization; writing—original draft.

CONFLICTS OF INTEREST

The authors declare no conflicts of interest.

TRANSPARENCY STATEMENT

The lead author Mitra Montazeri affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.

APPENDIX 1: KEYWORDS USED TO IDENTIFY RELEVANT PAPERS

PubMed

image

Web of Science

image

Scopus

image

We included original studies. The search was restricted to English-language publications. Editorials, commentaries, letters, books, presentations, and conference papers were excluded. All types of review studies were also excluded to prevent duplication in data collection. We had no filter on the date.

APPENDIX 2: DATA EXTRACTION FORM

A B C D E F G H I J K L M
1 First author Year Data utilized What is predicted? Disorder Sample size Machine learning model Accuracy (%) Sensitivity (%) Specificity (%) AUC (%) Precision (%) Risk of bias
2 Taylor et al.34 2017 EEG having Schizophrenia Schizophrenia 30 SVM 78.05 NR NR NR NR High

APPENDIX 3: MORE DETAILED TABLE OF THE PROBAST RESULTS

Study Risk of bias
Participants Predictors Outcome Analysis Overall
Taylor et al.37 Unclear Low Low High High
M. Karrer et al.43 Low Low Low High High
Talpalaru et al.44 Low Low Low Low Low
Zarogianni et al.38 Low Low Low Low Low
Hillegers et al.39 High Unclear Unclear High High
Pierrefeu et al.46 Low Low Low Low Low
Bracher-Smith et al.41 High Unclear Unclear High High
Fond et al.24 Low Low Low Low Low
Trakadis et al.42 Low Low Low Low Low
Salvador et al.29 Unclear Low Low Unclear Unclear
Nimkar et al.26 Low Low Low Low Low
Kalmady et al.47 Low Low Low Low Low
Sartori et al.30 Unclear Low Low High High
de Pierrefeu et al.40 Low Low Low Low Low
Silva et al.74 Low Low Low High High

DATA AVAILABILITY STATEMENT

The authors confirm that the data supporting the findings of this study are available within the article or its supplementary materials.

    The full text of this article hosted at iucr.org is unavailable due to technical difficulties.