Self-Paced (Asynchronous) BCI Control of a Wheelchair in Virtual Environments: A Case Study with a Tetraplegic
Corresponding Author
Robert Leeb
Laboratory of Brain-Computer Interfaces, Institute for Knowledge Discovery, Graz University of Technology, Krenngasse 37, Graz 8010, Austria tugraz.at
Search for more papers by this authorDoron Friedman
Department of Computer Science, University College London, Gower Street, London WC1E 6BT, UK ucl.ac.uk
Sammy Ofer School of Communications, The Interdisciplinary Center, P.O. Box 167, Herzliya 08010, Israel idc.ac.il
Search for more papers by this authorGernot R. Müller-Putz
Laboratory of Brain-Computer Interfaces, Institute for Knowledge Discovery, Graz University of Technology, Krenngasse 37, Graz 8010, Austria tugraz.at
Search for more papers by this authorReinhold Scherer
Laboratory of Brain-Computer Interfaces, Institute for Knowledge Discovery, Graz University of Technology, Krenngasse 37, Graz 8010, Austria tugraz.at
Search for more papers by this authorMel Slater
Department of Computer Science, University College London, Gower Street, London WC1E 6BT, UK ucl.ac.uk
Catalan Institute of Research and Advanced Studies (ICREA), Polytechnic University of Catalunya, Barcelona 08010, Spain upc.edu
Search for more papers by this authorGert Pfurtscheller
Laboratory of Brain-Computer Interfaces, Institute for Knowledge Discovery, Graz University of Technology, Krenngasse 37, Graz 8010, Austria tugraz.at
Search for more papers by this authorCorresponding Author
Robert Leeb
Laboratory of Brain-Computer Interfaces, Institute for Knowledge Discovery, Graz University of Technology, Krenngasse 37, Graz 8010, Austria tugraz.at
Search for more papers by this authorDoron Friedman
Department of Computer Science, University College London, Gower Street, London WC1E 6BT, UK ucl.ac.uk
Sammy Ofer School of Communications, The Interdisciplinary Center, P.O. Box 167, Herzliya 08010, Israel idc.ac.il
Search for more papers by this authorGernot R. Müller-Putz
Laboratory of Brain-Computer Interfaces, Institute for Knowledge Discovery, Graz University of Technology, Krenngasse 37, Graz 8010, Austria tugraz.at
Search for more papers by this authorReinhold Scherer
Laboratory of Brain-Computer Interfaces, Institute for Knowledge Discovery, Graz University of Technology, Krenngasse 37, Graz 8010, Austria tugraz.at
Search for more papers by this authorMel Slater
Department of Computer Science, University College London, Gower Street, London WC1E 6BT, UK ucl.ac.uk
Catalan Institute of Research and Advanced Studies (ICREA), Polytechnic University of Catalunya, Barcelona 08010, Spain upc.edu
Search for more papers by this authorGert Pfurtscheller
Laboratory of Brain-Computer Interfaces, Institute for Knowledge Discovery, Graz University of Technology, Krenngasse 37, Graz 8010, Austria tugraz.at
Search for more papers by this authorAbstract
The aim of the present study was to demonstrate for the first time that brain waves can be used by a tetraplegic to control movements of his wheelchair in virtual reality (VR). In this case study, the spinal cord injured (SCI) subject was able to generate bursts of beta oscillations in the electroencephalogram (EEG) by imagination of movements of his paralyzed feet. These beta oscillations were used for a self-paced (asynchronous) brain-computer interface (BCI) control based on a single bipolar EEG recording. The subject was placed inside a virtual street populated with avatars. The task was to “go” from avatar to avatar towards the end of the street, but to stop at each avatar and talk to them. In average, the participant was able to successfully perform this asynchronous experiment with a performance of 90%, single runs up to 100%.
References
- 1 Holden M. K., Virtual environments for motor rehabilitation: review, Cyberpsychology & Behavior. (2005) 8, no. 3, 187–211, https://doi.org/10.1089/cpb.2005.8.187.
- 2 Broeren J. and Scott O., Commentary on Holden, M. K., virtual environments for motor rehabilitation: review, CyberPsychology & Behavior. (2005) 8, no. 3, 212–219, https://doi.org/10.1089/cpb.2005.8.212.
- 3 Jack D., Boian R., Merians A. S. et al., Virtual reality-enhanced stroke rehabilitation, IEEE Transactions on Neural Systems and Rehabilitation Engineering. (2001) 9, no. 3, 308–318, https://doi.org/10.1109/7333.948460.
- 4 Girone M., Burdea G., Bouzit M., Popescu V., and Deutsch J. E., Orthopedic rehabilitation using the “rutgers ankle” interface, Studies in Health Technology and Informatics. (2000) 70, 89–95.
- 5 Webster J. S., McFarland P. T., Rapport L. J., Morrill B., Roades L. A., and Abadee P. S., Computer-assisted training for improving wheelchair mobility in unilateral neglect patients, Archives of Physical Medicine and Rehabilitation. (2001) 82, no. 6, 769–775, https://doi.org/10.1053/apmr.2001.23201.
- 6 Kenyon R. V. and Afenya M. B., Training in virtual and real environments, Annals of Biomedical Engineering. (1995) 23, no. 4, 445–455, https://doi.org/10.1007/BF02584444.
- 7 Rose F. D., Attree E. A., Brooks B. M. et al., Training in virtual environments: transfer to real world tasks and equivalence to real task training, Ergonomics. (2000) 43, no. 4, 494–511, https://doi.org/10.1080/001401300184378.
- 8 Todorov E., Shadmehr R., and Bizzi E., Augmented feedback presented in a virtual environment accelerates learning of a difficult motor task, Journal of Motor Behavior. (1997) 29, no. 2, 147–158.
- 9 Rose F. D., Brooks B. M., and Rizzo A. A., Virtual reality in brain damage rehabilitation: review, Cyberpsychology & Behavior. (2005) 8, no. 3, 241–262, https://doi.org/10.1089/cpb.2005.8.241.
- 10 Foreman N. and Stirk J., Commentary on Rose, F. D., Brooks, B. M. & Rizzo, A. A., virtual reality in brain damage rehabilitation: review, CyberPsychology & Behavior. (2005) 8, no. 3, 263–271, https://doi.org/10.1089/cpb.2005.8.263.
- 11 Pfurtscheller G. and Neuper C., Motor imagery and direct brain-computer communication, Proceedings of the IEEE. (2001) 89, no. 7, 1123–1134, https://doi.org/10.1109/5.939829.
- 12 Wolpaw J. R., Birbaumer N., McFarland D. J., Pfurtscheller G., and Vaughan T. M., Brain-computer interfaces for communication and control, Clinical Neurophysiology. (2002) 113, no. 6, 767–791, https://doi.org/10.1016/S1388-2457(02)00057-3.
- 13 Lalor E. C., Kelly S. P., Finucane C. et al., Steady-state VEP-based brain-computer interface control in an immersive 3D gaming environment, EURASIP Journal on Applied Signal Processing. (2005) 2005, no. 19, 3156–3164, https://doi.org/10.1155/ASP.2005.3156.
- 14 Bayliss J. D., Use of the evoked potential P3 component for control in a virtual apartment, IEEE Transactions on Neural Systems and Rehabilitation Engineering. (2003) 11, no. 2, 113–116, https://doi.org/10.1109/TNSRE.2003.814438.
- 15 Leeb R., Keinrath C., Friedman D. et al., Walking by thinking: the brainwaves are crucial, not the muscles!, Presence: Teleoperators and Virtual Environments. (2006) 15, no. 5, 500–514, https://doi.org/10.1162/pres.15.5.500.
- 16 Pfurtscheller G., Leeb R., Keinrath C. et al., Walking from thought, Brain Research. (2006) 1071, no. 1, 145–152, https://doi.org/10.1016/j.brainres.2005.11.083.
- 17 Borisoff J. F., Mason S. G., Bashashati A., and Birch G. E., Brain-computer interface design for asynchronous control applications: improvements to the LF-ASD asynchronous brain switch, IEEE Transactions on Biomedical Engineering. (2004) 51, no. 6, 985–992, https://doi.org/10.1109/TBME.2004.827078.
- 18 Mason S. G., Kronegg J., Huggins J. E., Fatourechi M., and Schlögl A., Evaluating the performance of self-paced brain-computer interface technology, 2006, Brain-Interface Laboratory, Neil Squire Society, Vancouver, Canada, http://www.bci-info.org/.
- 19 Millán J. R. and Mouriño J., Asynchronous BCI and local neural classifiers: an overview of the adaptive brain interface project, IEEE Transactions on Neural Systems and Rehabilitation Engineering. (2003) 11, no. 2, 159–161, https://doi.org/10.1109/TNSRE.2003.814435.
- 20 Müller-Putz G. R., Scherer R., Pfurtscheller G., and Rupp R., EEG-based neuroprosthesis control: a step towards clinical practice, Neuroscience Letters. (2005) 382, no. 1-2, 169–174, https://doi.org/10.1016/j.neulet.2005.03.021.
- 21 Scherer R., Müller-Putz G. R., Neuper C., Graimann B., and Pfurtscheller G., An asynchronously controlled EEG-based virtual keyboard: improvement of the spelling rate, IEEE Transactions on Biomedical Engineering. (2004) 51, no. 6, 979–984, https://doi.org/10.1109/TBME.2004.827062.
- 22 Townsend G., Graimann B., and Pfurtscheller G., Continuous EEG classification during motor imagery-simulation of an asynchronous BCI, IEEE Transactions on Neural Systems and Rehabilitation Engineering. (2004) 12, no. 2, 258–265, https://doi.org/10.1109/TNSRE.2004.827220.
- 23 Pfurtscheller G., Guger C., Müller-Putz G. R., Krausz G., and Neuper C., Brain oscillations control hand orthosis in a tetraplegic, Neuroscience Letters. (2000) 292, no. 3, 211–214, https://doi.org/10.1016/S0304-3940(00)01471-3.
- 24 Müller-Putz G. R., Scherer R., Pfurtscheller G., and Rupp R., Brain-computer interfaces for control of neuroprostheses: from synchronous to asynchronous mode of operation, Biomedizinische Technik. (2006) 51, no. 2, 57–63, https://doi.org/10.1515/BMT.2006.011.
- 25 Pfurtscheller G., Müller-Putz G. R., Pfurtscheller J., Gerner H. J., and Rupp R., ‘Thought’—control of functional electrical stimulation to restore hand grasp in a patient with tetraplegia, Neuroscience Letters. (2003) 351, no. 1, 33–36, https://doi.org/10.1016/S0304-3940(03)00947-9.
- 26 Jasper H. H., The ten-twenty electrode system of the international federation, Electroencephalography and Clinical Neurophysiology. (1958) 10, no. 2, 370–375.
- 27 Pfurtscheller G., Müller-Putz G. R., Schlögl A. et al., 15 years of BCI research at Graz university of technology: current projects, IEEE Transactions on Neural Systems and Rehabilitation Engineering. (2006) 14, no. 2, 205–210, https://doi.org/10.1109/TNSRE.2006.875528.
- 28 rtsBCI, Graz brain-computer interface real-time open source package, http://sourceforge.net/projects/biosig/, 2004-2007.
- 29 BioSig, An open source software package for biomedical signal processing under matlab, http://biosig.sf.net/, 2003-2007.
- 30 Cruz-Neira C., Sandin D. J., and DeFanti T. A., Surround-screen projection-based virtual reality: the design and implementation of the CAVE, Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques, August 1993, Anaheim, Calif, USA, 135–142, https://doi.org/10.1145/166117.166134.
- 31 Slater M., Steed A., and Chrysanthou Y., Computer Graphics and Virtual Environments: From Realism to Real-Time, 2001, Addison-Wesley, Boston, Mass, USA.
- 32 Frécon E., Smith G., Steed A., Stenius M., and Ståhl O., An overview of the COVEN platform, Presence: Teleoperators and Virtual Environments. (2001) 10, no. 1, 109–127, https://doi.org/10.1162/105474601750182351.
- 33 VRPN, Virtual reality peripheral network, http://www.cs.unc.edu/Research/vrpn/.
- 34 Scherer R., Schlögl A., Lee F., Bischof H., Grassi D., and Pfurtscheller G., The self-paced Graz brain-computer interface: methods and applications, to appear in Computational Intelligence and Neuroscience, 2007.
- 35 Millán J. R., Renkens F., Mouriño J., and Gerstner W., Noninvasive brain-actuated control of a mobile robot by human EEG, IEEE Transactions on Biomedical Engineering. (2004) 51, no. 6, 1026–1033, https://doi.org/10.1109/TBME.2004.827086.
