SPECIAL ISSUE PAPER
Deep learning for EEG data analytics: A survey
Gen Li,
Chang Ha Lee,
Jason J. Jung,
Young Chul Youn,
David Camacho,
Gen Li
Department of Computer Engineering, Chung-Ang University, Seoul, South Korea
Search for more papers by this authorChang Ha Lee
Department of Computer Engineering, Chung-Ang University, Seoul, South Korea
Search for more papers by this authorCorresponding Author
Jason J. Jung
Department of Computer Engineering, Chung-Ang University, Seoul, South Korea
Jason J. Jung, Department of Computer Engineering, Chung-Ang University, Seoul 06974, South Korea.
Email: [email protected]
Search for more papers by this authorYoung Chul Youn
Neurology, Chung-Ang University Hospital, Seoul, South Korea
Search for more papers by this authorDavid Camacho
Department of Computer Engineering, Universidad Autonoma de Madrid, Madrid, Spain
Search for more papers by this authorGen Li,
Chang Ha Lee,
Jason J. Jung,
Young Chul Youn,
David Camacho,
Gen Li
Department of Computer Engineering, Chung-Ang University, Seoul, South Korea
Search for more papers by this authorChang Ha Lee
Department of Computer Engineering, Chung-Ang University, Seoul, South Korea
Search for more papers by this authorCorresponding Author
Jason J. Jung
Department of Computer Engineering, Chung-Ang University, Seoul, South Korea
Jason J. Jung, Department of Computer Engineering, Chung-Ang University, Seoul 06974, South Korea.
Email: [email protected]
Search for more papers by this authorYoung Chul Youn
Neurology, Chung-Ang University Hospital, Seoul, South Korea
Search for more papers by this authorDavid Camacho
Department of Computer Engineering, Universidad Autonoma de Madrid, Madrid, Spain
Search for more papers by this authorSummary
In this work, we conducted a literature review about deep learning (DNN, RNN, CNN, and so on) for analyzing EEG data for decoding the activity of human's brain and diagnosing disease and explained details about various architectures for understanding the details of CNN and RNN. It has analyzed a word, which presented a model based on CNN and LSTM methods, and how these methods can be used to both optimize and set up the hyper parameters of deep learning architecture. Later, it is studied how semi-supervised learning on EEG data analytics can be applied. We review some studies about different methods of semi-supervised learning on EEG data analytics and discussing the importance of semi-supervised learning for analyzing EEG data. In this paper, we also discuss the most common applications for human EEG research and review some papers about the application of EEG data analytics such as Neuromarketing, human factors, social interaction, and BCI. Finally, some future trends of development and research in this area, according to the theoretical background on deep learning, are given.
REFERENCES
- 1Manyika J, Chui M, Brown B, et al. Big data: the next frontier for innovation, competition, and productivity. New York, NY: McKinsey Global Institute; 2011.
- 2Min S, Lee B, Yoon S. Deep learning in bioinformatics. Brief Bioinform. 2017; 18(5): 851-869.
- 3Zhao G, Liu S, Wang Q, et al. Deep convolutional neural network for drowsy student state detection. Concurrency Computat Pract Exper. 2018; 30(23):e4457.
- 4Liu J-C, Leu F-Y, Lin G-L, Susanto H. An MFCC-based text-independent speaker identification system for access control. Concurrency Computat Pract Exper. 2018; 30(2):e4255.
- 5Ferrucci D, Brown E, Chu-Carroll J, et al. Building Watson: an overview of the deep QA project. AI Mag. 2010; 31(3): 59-79.
- 6 IBM. Oncology IBM Watson. 2018. http://www.ibm.com/
- 7Hinton GE, Osindero S, Teh Y-W. A fast learning algorithm for deep belief nets. Neural Comput. 2006; 18(7): 1527-1554.
- 8van Etten Jaap. Datenna. 2017. http://www.datenna.com/industry/china-marching-forward-with-artificial-intelligence
- 9LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015; 521(7553): 436-444.
- 10Farabet C, Couprie C, Najman L, LeCun Y. Learning hierarchical features for scene labeling. IEEE Trans Pattern Anal Mach Intell. 2013; 35(8): 1915-1929.
- 11Szegedy C, Liu W, Jia Y, et al. Going deeper with convolutions. Paper presented at: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2015; Boston, MA.
- 12Chorowski JK, Bahdanau D, Serdyuk D, Cho K, Bengio Y. Attention-based models for speech recognition. In: Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1 (NIPS); 2015; Montreal, Canada.
- 13Luong M-T, Pham H, Manning CD. Effective approaches to attention-based neural machine translation. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing; 2015; Lisbon, Portugal.
- 14Guan Z, Chen L, Zhao W, Zheng Y, Tan S, Cai D. Weakly-supervised deep learning for customer review sentiment classification. In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI); 2016; New York, NY.
- 15Stober S, Cameron DJ, Grahn JA. Using convolutional neural networks to recognize rhythm stimuli from electroencephalography recordings. In: Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 1 (NIPS); 2014; Montreal, Canada.
- 16Cecotti H, Graser A. Convolutional neural networks for P300 detection with application to brain-computer interfaces. IEEE Trans Pattern Anal Mach Intell. 2011; 33(3): 433-445.
- 17Tang Z, Li C, Sun S. Single-trial EEG classification of motor imagery using deep convolutional neural networks. Optik-Int J Light Electron Opt. 2017; 130: 11-18.
- 18Lawhern VJ, Solon AJ, Waytowich NR, Gordon SM, Hung CP, Lance BJ. EEGNet: a compact convolutional network for EEG-based brain-computer interfaces. arXiv preprint arXiv:1611.08024. 2016.
- 19Sun X, Qian C, Chen Z, Wu Z, Luo B, Pan G. Remembered or forgotten?– An EEG-based computational prediction approach. PLoS One. 2016; 11(12):e0167497.
- 20Chang J, Gu Y, Zhang Y. Chinese typeface transformation with hierarchical adversarial network. arXiv preprint arXiv:1711.06448. 2017.
- 21Goodfellow I, Bengio Y, Courville A. Deep Learning. Cambridge, MA: MIT Press; 2016.
- 22Wang X-W, Nie D, Lu B-L. Emotional state classification from EEG data using machine learning approach. Neurocomputing. 2014; 129: 94-106.
- 23Müller, K-R, Tangermann M, Dornhege G, Krauledat M, Curio G, Blankertz B. Machine learning for real-time single-trial EEG-analysis: from brain–computer interfacing to mental state monitoring. J Neurosci Methods. 2008; 167(1): 82-90.
- 24Shoeb AH, Guttag JV. Application of machine learning to epileptic seizure detection. In: Proceedings of the 27th International Conference on International Conference on Machine Learning (ICML); 2010; Haifa, Israel.
- 25Jia X, Li K, Li X, Zhang A. A novel semi-supervised deep learning framework for affective state recognition on EEG signals. In: Proceedings of the 2014 IEEE International Conference on Bioinformatics and Bioengineering (BIBE); 2014; Boca Raton, FL.
- 26Jirayucharoensak S, Pan-Ngum S, Israsena P. EEG-based emotion recognition using deep learning network with principal component based covariate shift adaptation. Sci World J. 2014; 2014.
- 27Zheng W-L, Guo H-T, Lu B-L. Revealing critical channels and frequency bands for emotion recognition from EEG with deep belief network. In: Proceedings of the 7th International IEEE/EMBS Conference on Neural Engineering (NER); 2015; Montpellier, France.
- 28An X, Kuang D, Guo X, Zhao Y, He L. A deep learning method for classification of EEG data based on motor imagery. In: Intelligent Computing in Bioinformatics: 10th International Conference, ICIC 2014, Taiyuan, China, August 3-6, 2014. Proceedings. Cham, Switzerland: Springer International Publishing; 2014: 203-210.
10.1007/978-3-319-09330-7_25 Google Scholar
- 29Wulsin DF, Gupta JR, Mani R, Blanco JA, Litt B. Modeling electroencephalography waveforms with semi-supervised deep belief nets: fast classification and anomaly measurement. J Neural Eng. 2011; 8(3): 036015.
- 30Turner JT, Page A, Mohsenin T, Oates T. Deep belief networks used on high resolution multichannel electroencephalography data for seizure detection. Paper presented at: 2014 AAAI Spring Symposium Series; 2014; Palo Alto, CA. https://arxiv.org/abs/1708.08430
- 31Zhao Y, He L. Deep Learning in the EEG diagnosis of Alzheimer's disease. In: Computer Vision - ACCV 2014 Workshops. Cham, Switzerland: Springer International Publishing; 2014: 340-353.
- 32Längkvist M, Karlsson L, Loutfi A. Sleep stage classification using unsupervised feature learning. Adv Artif Neural Syst. 2012; 2012.
- 33Hajinoroozi M, Jung T-P, Lin C-T, Huang Y. Feature extraction with deep belief networks for driver's cognitive states prediction from EEG data. Paper presented at: 2015 IEEE China Summit and International Conference on Signal and Information Processing (ChinaSIP); 2015; Chengdu, China.
- 34Petrosian A, Prokhorov D, Homan R, Dasheiff R, Wunsch II D. Recurrent neural network based prediction of epileptic seizures in intra- and extracranial EEG. Neurocomputing. 2000; 30(1-4): 201-218.
- 35Davidson PR, Jones RD, Peiris MTR. EEG-based lapse detection with high temporal resolution. IEEE Trans Biomed Eng. 2007; 54(5): 832-839.
- 36Minasyan GR, Chatten JB, Chatten MJ, Harner RN. Patient-specific early seizure detection from scalp EEG. J Clin Neurophysiol. 2010; 27(3): 163-178.
- 37Naderi MA, Mahdavi-Nasab H. Analysis and classification of EEG signals using spectral analysis and recurrent neural networks. Paper presented at: 2010 17th Iranian Conference of Biomedical Engineering (ICBME); 2010; Isfahan, Iran.
- 38Soleymani M, Asghari-Esfeden S, Pantic M, Fu Y. Continuous emotion detection using EEG signals and facial expressions. Paper presented at: 2014 IEEE International Conference on Multimedia and Expo (ICME); 2014; Chengdu, China.
- 39Forney EM, Anderson CW. Classification of EEG during imagined mental tasks by forecasting with Elman recurrent neural networks. Paper presented at: The 2011 International Joint Conference on Neural Networks; 2011; San Jose, CA.
- 40Li Y, Guan C, Li H, Chin Z. A self-training semi-supervised SVM algorithm and its application in an EEG-based brain computer interface speller system. Pattern Recogn Lett. 2008; 29(9): 1285-1294.
- 41Patnaik S, Moharkar L, Chaudhari A. Deep RNN learning for EEG based functional brain state inference. Paper presented at: 2017 International Conference on Advances in Computing, Communication and Control (ICAC3); 2017; Mumbai, India.
- 42Ni Z, Yuksel AC, Ni X, Mandel MI, Xie L. Confused or not confused?: disentangling brain activity from EEG data using bidirectional LSTM recurrent neural networks. In: Proceedings of the 8th ACM International Conference on Bioinformatics, Computational Biology,and Health Informatics (ACM-BCB); 2017; Boston, MA.
- 43Thodoroff P, Pineau J, Lim A. Learning robust features using deep learning for automatic seizure detection. In: Proceedings of the 1st Machine Learning for Healthcare Conference; 2016; Los Angeles, CA.
- 44Shamwell J, Lee H, Kwon H, Marathe AR, Lawhern V, Nothwang W. Single-trial EEG RSVP classification using convolutional neural networks. In: Proceedings of the SPIE Volume 9836, Micro- and Nanotechnology Sensors, Systems, and Applications VIII; 2016; Baltimore, MD.
- 45Manor R, Geva AB. Convolutional neural network for multi-category rapid serial visual presentation BCI. Front Comput Neurosci. 2015; 9: 146.
- 46Sakhavi S, Guan C, Yan S. Parallel convolutional-linear neural network for motor imagery classification. Paper presented at: 2015 23rd European Signal Processing Conference (EUSIPCO); 2015; Nice, France.
- 47Ren Y, Wu Y. Convolutional deep belief networks for feature extraction of EEG signal. Paper presented at: 2014 International Joint Conference on Neural Networks (IJCNN); 2014; Beijing, China.
- 48Ruffini G, Soria DI, Dubreuil L, Gagnon J-F, Montplaisir J, Soria-Frisch A. Deep learning using EEG spectrograms for prognosis in idiopathic rapid eye movement behavior disorder (RBD). bioRxiv. 2018. https://doi.org/10.1101/240267
- 49Hajinoroozi M, Zhang JM, Huang Y. Driver's fatigue prediction by deep covariance learning from EEG. Paper presented at: 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC); 2017; Banff, Canada.
- 50Jiao Z, Gao X, Wang Y, Li J, Xu H. Deep convolutional neural networks for mental load classification based on EEG data. Pattern Recogn. 2018; 76: 582-595.
- 51Mirowski P, Madhavan D, LeCun Y, Kuzniecky R. Classification of patterns of EEG synchronization for seizure prediction. Clin Neurophysiol. 2009; 120(11): 1927-1940.
- 52Liang J, Lu R, Wang F, Zhang C. Predicting seizures from electroencephalography recordings: a knowledge transfer strategy. Paper presented at: 2016 IEEE International Conference on Healthcare Informatics (ICHI); 2016; Chicago, IL.
- 53Antoniades A, Spyrou L, Took CC, Sanei S. Deep learning for epileptic intracranial EEG data. Paper presented at: 2016 IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP); 2016; Vietri sul Mare, Italy.
- 54Page A, Shea C, Mohsenin T. Wearable seizure detection using convolutional neural networks with transfer learning. Paper presented at: 2016 IEEE International Symposium on Circuits and Systems (ISCAS); 2016; Montreal, Canada.
- 55Acharya UR, Oh SL, Hagiwara Y, Tan J-H, Adeli H. Deep convolutional neural network for the automated detection and diagnosis of seizure using EEG signals. Comput Biol Med. 2018; 100: 270-278.
- 56Wulsin D, Blanco J, Mani R, Litt B. Semi-supervised anomaly detection for EEG waveforms using deep belief nets. Paper presented at: 2010 Ninth International Conference on Machine Learning and Applications; 2010; Washington, DC.
- 57Shi L-C, Yu H, Lu B-L. Semi-supervised clustering for vigilance analysis based on EEG. Paper presented at: 2007 International Joint Conference on Neural Networks; 2007; Orlando, FL.
- 58Bashivan P, Rish I, Yeasin M, Codella N. Learning representations from EEG with deep recurrent-convolutional neural networks. arXiv preprint arXiv:1511.06448. 2015.
- 59Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput. 1997; 9(8): 1735-1780.
- 60He K, Zhang X, Ren S, Sun J. Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. In: Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV); 2015; Santiago, Chile.
- 61Saxe AM, McClelland JL, Ganguli S. Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. arXiv preprint arXiv:1312.6120. 2013.
- 62Bottou L. Large-scale machine learning with stochastic gradient descent. In: Proceedings of COMPSTAT'2010. Berlin, Germany: Springer-Verlag Berlin Heidelberg; 2010: 177-186.
10.1007/978-3-7908-2604-3_16 Google Scholar
- 63Bottou L, Curtis FE, Nocedal J. Optimization methods for large-scale machine learning. SIAM Rev. 2018; 60(2): 223-311.
- 64Kingma DP, Ba J. Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980. 2014.
- 65Yadava M, Kumar P, Saini R, Roy PP, Dogra DP. Analysis of EEG signals and its application to neuromarketing. Multimed Tools Appl. 2017; 76(18): 19087-19111.
- 66Murugappan M, Murugappan S, Gerard C. Wireless EEG signals based neuromarketing system using fast fourier transform (FFT). Paper presented at: 2014 IEEE 10th International Colloquium on Signal Processing and its Applications; 2014; Kuala Lumpur, Malaysia.
- 67Gevins A, Smith ME, Leong H, et al. Monitoring working memory load during computer-based tasks with EEG pattern recognition methods. Hum Factors. 1998; 40(1): 79-91.
- 68Smith ME, Gevins A, Brown H, Karnik A, Du R. Monitoring task loading with multivariate EEG measures during complex forms of human-computer interaction. Hum Factors. 2001; 43(3): 366-380.
- 69Perry A, Stein L, Bentin S. Motor and attentional mechanisms involved in social interaction—evidence from mu and alpha EEG suppression. Neuroimage. 2011; 58(3): 895-904.
- 70Lotte F, Congedo M, Lécuyer A, Lamarche F, Arnaldi B. A review of classification algorithms for EEG-based brain–computer interfaces. J Neural Eng. 2007; 4(2): R1.
- 71Pfurtscheller G, Neuper C, Flotzinger D, Pregenzer M. EEG-based discrimination between imagination of right and left hand movement. Electroencephalogr Clin Neurophysiol. 1997; 103(6): 642-651.
- 72Chiappa S, Bengio S. HMM and IOHMM Modeling of EEG Rhythms for Asynchronous BCI Systems. Research Report. Martigny, Switzerland: IDIAP; 2003.
- 73Millan JR, Mouriño J. Asynchronous BCI and local neural classifiers: an overview of the adaptive brain interface project. IEEE Trans Neural Syst Rehabil Eng. 2003; 11(2): 159-161.
- 74Lehmann C, Koenig T, Jelic V, et al. Application and comparison of classification algorithms for recognition of Alzheimer's disease in electrical brain activity (EEG). J Neurosci Methods. 2007; 161(2): 342-350.
- 75Simpraga S, Alvarez-Jimenez R, Mansvelder HD, et al. EEG machine learning for accurate detection of cholinergic intervention and Alzheimer's disease. Sci Rep. 2017; 7(1): 5775.
- 76Goli P, Rad EM, Ghandehari K, Azarnoosh M. Early assessment of mild Alzheimer's disease using Elman neural network, LDA and SVM methods. Mach Learn Res. 2017; 2(4): 148-151.
- 77Trambaiolli LR, Lorena AC, Fraga FJ, Kanda PAM, Anghinah R, Nitrini R. Improving Alzheimer's disease diagnosis with machine learning techniques. Clin EEG Neurosci. 2011; 42(3): 160-165.
- 78Song Y, Crowcroft J, Zhang J. Automatic epileptic seizure detection in EEGs based on optimized sample entropy and extreme learning machine. J Neurosci Methods. 2012; 210(2): 132-146.
- 79Tzimourta KD, Tzallas AT, Giannakeas N, Astrakas LG, Tsalikakis DG, Tsipouras MG. Epileptic seizures classification based on long-term EEG signal wavelet analysis. In: N Maglaveras, I Chouvarda, P Carvalho, eds. Precision Medicine Powered by pHealth and Connected Health. Singapore: Springer Nature Singapore Pte Ltd; 2018: 165-169. IFMBE Proceedings; vol. 66.
10.1007/978-981-10-7419-6_28 Google Scholar
- 80Golmohammadi M, Torbati AHHN, Lopez de Diego S, Obeid I, Picone J. Automatic analysis of EEGs using big data and hybrid deep learning architectures. arXiv preprint arXiv:1712.09771. 2017.
- 81Tsiouris KM, Pezoulas VC, Zervakis M, Konitsiotis S, Koutsouris DD, Fotiadis DI. A long short-term memory deep learning network for the prediction of epileptic seizures using EEG signals. Comput Biol Med. 2018; 99: 24-37.
Citing Literature
September 25, 2020
e5199
