This article presents a systematic review of up-to-date progress on upper limb and lower extremity exoskeletons and prosthetic devices intuitively controlled by electromyography (EMG) or electroencephalography (EEG) signals for paraplegic patients and amputees. Bio-electrical signal processing methods are introduced, including waveform recognition and template matching (WRTM), wavelet analysis, fast Fourier transform (FFT), static status evoked visual potential (SSEVP) methods. Progress in electrodes used as human machine interface (HMI) is introduced, including carbon nanotube multi-electrode array (CNT-MEA).

Keywords: robotics, rehabilitation, electromyography, electroencephalography, human machine interface, electrode

BCI, brain computer interface; BMI, brain machine interface; EMG, electromyography; EEG, electroencephalography; ECoG, electrocorticography; ERP, event related potential; HMI, human machine interface; PNI, peripheral nerve interface; PMI, peripheral muscle interface; CNT-MEA, carbon nano tube multi-electrode array; WRTM, waveform recognition and template matching; FFT, fast fourier transform; SSEVP, static status evoked visual potential: SNR, signal-to-noise ratio; VR, virtual reality

Background

Robotic systems for rehabilitation focus on regaining missing extremity motor and sensation. Intuitive robotic prosthetics and exoskeletons aim at providing dexterity, natural mobility and sense of touch to missing or paralyzed limbs.

Purpose

The main objective of our work is to develop a human machine interface (HMI) that captures high bio-fidelity bioelectrical signals including EMG and EEG signals using novel nano-graphene electrodes, also named as carbon nanotube multi-electrode arrays (CNT-MEA). Waveform recognition and template matching (WRTM), fast Fourier transform (FFT), static status evoked visual potential (SSEVP) techniques are used for bio-electrical signal processing. These bioelectrical signals are decoded and encoded for making algorithms to intuitively control any exoskeletons and prosthetic devices. The intuitive robotic control system will transform thought and muscular activity into actions of exoskeleton and prosthesis, which enable paralyzed people or amputees to better control prosthetic limbs or exoskeletons and return to normal daily life.

Electrodes for human machine interface

Bioelectrical signals are mainly the result of the electrochemical activity of the nervous and muscular tissue. The neural activity of the brain cells can be monitored on the scalp and this signal is called electroencephalography (EEG) and in the cortex of brain called electrocorticography (ECoG). The electromyography (EMG) signal is a measure of muscle activity.¹ Muscle contraction is the result of activation of a number of muscle fibers. This process of activation generates a change in electrical potential and can be measured in sum at the surface of the skin as a surface electromyogram (sEMG). The amplitude of the EMG signal is directly correlated with the force generated by the muscle,² which can be recorded on skin surface or intramuscularly. The latest developments in invasive and noninvasive human-machine interfaces (HMIs) include invasive electrodes that are inserted inside the brain and muscles and non-invasive surface EMG and EEG electrodes.

Conventional surface gel electrodes are widely used for bioelectrical signal recording. However, long term application of the electrode causes skin irritation and other problems, including inconvenience in set-up and fluctuation of EMG signals. Dry electrodes have been introduced using rigid metal pins, which solve some of wet electrode’s problems; however, their rigidity may cause discomfort and pain. Recently, soft polymer dry electrodes have been introduced for EEG recording³ by using ethylene propylene diene monomer (EPDM) rubber containing various carbon contents additives. The skin-electrode impedance is approximately 10 times larger than that of gel electrodes, demonstrating a better efficiency in clinical trials.³

There are 3 major categories of implantable multi-electrode arrays (MEAs), including microwire, silicon- based electrodes, and flexible microelectrode arrays. Microwire MEAs are largely made of stainless steel or tungsten and they can be used to estimate the position of individual recorded neurons by triangulation. Silicon-based microelectrode arrays⁴ include two specific models: the Michigan⁵ and Utah arrays.⁴ Michigan arrays allow a higher density of sensors for implantation as well as a higher spatial resolution than microwire MEAs. They also allow signals to be obtained along the length of the shank, rather than just at the ends of the shanks. Utah arrays are 3-Dimesional electrode arrays, consisting of 100 conductive silicon needles. A Utah array signals are only received from the tips of each electrode, which limits the amount of information that can be obtained at one time. Recently, Waterloo flexible arrays made with polymers, have been introduced, which exhibited a remarkable drop in the electric impedance (100 times at 100 Hz), this improves electrode-electrolyte interface noise producing a higher signal-to-noise ratio (SNR) (3.3 times).⁶ In our laboratory, a simple microfabrication process has been used to fabricate the high quality vertical CNTs electrode.⁷ Compared to a platinum (Pt) electrode, the impedance of the CNT electrode was significantly lowered than that of the Pt electrode, particularly at the frequencies below 10 kHz. The impedance of CNT electrodes is about 480 Ω at 1 kHz, about 10 times smaller than that of Pt electrodes. This CNT electrode was utilized for extracellular neuronal recording and stimulation in an animal model, the results demonstrated that the signal-to-noise ratio of the CNTs electrode was much higher than that of Pt electrode that was about 12:1.^7,8 All these MEAs can be used to record EMG and EEG signals.

EMG and EEG controlled medical robotic devices

EMG-controlled prosthesis: Use of electromyography (EMG) signals to intuitively control prostheses has been proposed for transradial and knee amputees for many years.^9,10 There are several methods using EMG signals to control prosthesis, including conventional methods, pattern recognition methods, direct extraction of neural code from sEMG and multi-model approaches. To date, only simple conventional control strategies have been implemented in commercial/clinical system which is only half-automatically controlled system,¹¹ because the conventional use of surface EMG or myoelectric controllers has been limited due to rapidly fluctuating surface EMG signals that slow down the responsivity of the controller . Surface EMG crosstalk also introduces an influence of each EMG signal on the other, increasing the noise ration and reducing the controllability. Pattern recognition methods have been applied for classifying EMG signals.¹² Pattern recognition methods are based on surface EMG signal processing for the maximum likelihood of user's intent, this increases classification accuracies in laboratory setting,¹³ yet classification accuracy is not completely satisfied.¹⁴

Electrocorticography (ECoG) controlled robotic arm: ECoG controlled robotic arm has been designed for people with tetraplegia to reach and grasp stuffs using a robotic arm. A neural interface of multi-electrode arrays (MEAs) can be implanted into patient’s brain to record neural activity of intent for robotic arm movement.¹⁵ Although robotic reach and grasp actions were not as fast or accurate as those of an able-bodied person, these studies demonstrated the feasibility for people with tetraplegia to recreate useful multidimensional control of complex devices directly from a small sample of neural signals.^15-17

EEG controlled lower limb exoskeleton: EEG has been proposed for control of lower limb exoskeleton to help paraplegic person rise from a wheelchair and walk several steps.^18,19 Long-term training with wearing a EEG controlled lower limb exoskeleton has showed a potential for partial neurological recovery in paraplegic patients, indicating that brain-machine interfaces (BMIs) may provide a new assistive strategy aimed at restoring mobility in severely paralyzed patients.^18,20 Technologies used for this purpose include wearing a custom-designed lower limb exoskeleton capable of delivering tactile feedback to subjects, combining virtual reality (VR) training and visual-tactile feedback and walking with two EEG-controlled exoskeleton. Clinical treatment showed that 50% of complete paraplegic patients can be upgraded to an incomplete paraplegia classification.¹⁸

Peripheral neural interface for hand prosthesis with sensation: Hand loss is a disabling disorder that significantly affects the quality of life. To achieve a natural replacement for the lost hand, a hand prosthesis with sensation has been proposed and tested in clinically.²¹ The novel prosthesis provides the user sensations when grasping or manipulating an object. Transversal intrafascicular multichannel electrodes (TIME)²² were utilized and inserted into the median nerve in their studies. The clinical trial results demonstrated that a high complexity of perception can be obtained, allowing the subject to identify the stiffness and shape of three different objects. However, over 4 weeks’ time period, perceptive sensation signals gradually traded off, with higher electrical current charges required to elicit sensation in fingers.²¹

SSVEP controlled medical devices: Static status evoked visual potential (SSEVP) is a kind of EEG signals elicited by flickering LED light stimulation. The SSVEP obtained by portable EEG recording can be encoded to control robot movements. Some experiments have demonstrated 75% accuracy for robot performance,²³ while other showed an accuracy of 91% and responding time of 3.5-6.5 seconds.²⁴ Our research group used oscillating Newton’s rings to elicit steady-state motion visual evoked potentials (SSMVEP), four motion reversal frequencies of 8.1, 9.8, 12.25 and 14 Hz were tested, with accuracy of detecting 4 distinct corresponding signals exceeded 80% with a responding time of 3.5 seconds.²⁵ By adding visual noise to the stimulating paradigm to boost SSMEVP, 40 distinct responding EEG frequency signals were detected with a successful rate of 86.25%±7.86% (range: 80-100%),²⁶ demonstrating that noise could boost event related potential (ERP) in addressing human needs in BCIs research. SSVEP has been encoded for spelling, the maximum spelling characters can be 40 words when waveform pattern recognition and template matching method was used. Accuracy of EEG based spelling ranged from 78.95% to 98%, with a speed of spelling at 60 characters per minutes.²⁷ Waveform recognition and template matching has been used in neural signal processing. It has been used to identify single sensory receptor’s responses to tissue stretch, which demonstrated an accuracy of 97% in identifying single neuron’s response.^28-32 This technology shows a promise in identifying and classifying neuronal activity more accurately.

Overall, the ability to control prosthesis in an intuitive manner still lags behind in scientific research. This limitation is partially due to the nature of human bioelectrical signals, for which the signal to noise ratio (SNR) is low when conventional electrodes are used. High noise makes it difficult to precisely control prosthesis motions. Hence, electrodes used for EMG and EEG recording and stimulation have been upgraded over past several decades. Electrodes used for peripheral neural interface include cuff electrode,³³ FINE,³⁴ TIME,²² LIFE,³⁵ USEA,⁴ sieve electrode,³⁶ microchannel electrode,³⁷ regenerative electrode³⁸ and CNT electrode.³⁹ Carbon nanotube (CNT) MEAs has been demonstrated as a promising material for neuronal interfaces. CNTs have major advantages owing to their unique mechanical and electrical properties. CNTs produce better neuronal cell adhesion and better signal resolution during multi electrode recordings. Advances in this field progress toward flexible, bio-compatible CNT-based electrodes.

Obtaining robust EMG-control information is difficult, because surface EMG signals are variable and noisy. In addition to using high bio-fidelity electrode, real-time filtering techniques may be used to alleviate this noise.¹⁰ This includes using the overall patterns of EMG activity for each ambulation mode. The low rates of classification errors indicate that the patterns of EMG activity are reliable and repeatable.

Using SSVEP/SSMVEP responding to an externally visual stimulus (VR) and generating command signals for the robot is a promising technology in EEG controlled robotic system. Fast Fourier Transform (FFT) or WRMT can be used to detect the dominant frequency component or neural responding patterns with high accuracy and short responding time.

This work is supported by the Science Fund of State Key Laboratory for Manufacturing Systems Engineering, Xian Jiaotong University, Xian, China (#sklms2016008).

There is not any financial interest or any conflict of interest for this work.

1. Giggins OM, Persson UM, Caulfield B. Biofeedback in rehabilitation. J Neuroeng Rehabil. 2013;10:60.
2. Shenoy KJP, Miller B, Crawford R, et al. Online electromyographic control of a robotic prosthesis. IEEE Trans Biomed Eng. 2008;55(3):1128–1135.
3. Che YH, Op de Beeck M, Vanderheyden L, et al. Soft, comfortable polymer dry electrodes for high quality ECG and EEG recording. Sensors (Basel). 2014;14(12):23758–23780.
4. Bhandari R, Negi S, Solzbacher F. Wafer-scale fabrication of penetrating neural microelectrode arrays. Biomed Microdevices. 2010;12(5):797–807.
5. Vetter RJ, Williams JC, Hetke JF, et al. Chronic neural recording using silicon-substrate microelectrode arrays implanted in cerebral cortex. IEEE Trans Biomed Eng. 2004;51(6):896–904.
6. Gabran SR, Salam MT, Dian J, et al. 3-D flexible nano-textured high-density microelectrode arrays for high-performance neuro-monitoring and neuro-stimulation. IEEE Trans Neural Syst Rehabil Eng. 2014;22(5):1072–1082.
7.  Yi W, Chen C, Feng Z, et al. A flexible and implantable microelectrode arrays using high-temperature grown vertical carbon nanotubes and a biocompatible polymer substrate. Nanotechnology. 2015;26(12):125301.
8. Chen C, Yi W, Meng X, et al. Comparisons of Platinum and CNT-MEA Electrodes as Peripheral Muscular Interface. In Annual Meeting of Biomedical Engineering, USA: Tampa; 2015.
9. Milligan J, Lee J, McMillan C, et al. Autonomic dysreflexia: recognizing a common serious condition in patients with spinal cord injury. Can Fam Physicia. 2012;58(8):831–835.
10. Hargrove LJ, Simon AM, Young AJ, et al. Robotic leg control with EMG decoding in an amputee with nerve transfers. N Engl J Med. 2013;369:1237–1242.
11. Farina D, Jiang N, Rehbaum H, et al. The extraction of neural information from the surface EMG for the control of upper-limb prostheses: emerging avenues and challenges. IEEE Trans Neural Syst Rehabil Eng. 2014;22(4):797–809.
12. Scheme E, Englehart K. Electromyogram pattern recognition for control of powered upper-limb prostheses: state of the art and challenges for clinical use. J Rehabil Res Dev. 2011;48(6):643–659.
13. Sebelius F, Eriksson L, Balkenius C, et al. Myoelectric control of a computer animated hand: a new concept based on the combined use of a tree-structured artificial neural network and a data glove. J Med Eng Technol. 2006;30(1):2–10.
14. Amsuss S, Goebel PM, Jiang N, et al. Self-correcting pattern recognition system of surface EMG signals for upper limb prosthesis control. IEEE Trans Biomed Eng. 2014;61(4):1167–1176.
15. Hochberg LR, Bacher D, Jarosiewicz B, et al. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature. 2012;485(7398):372–375.
16. Memberg WD, Polasek KH, Hart RL, et al. Implanted neuroprosthesis for restoring arm and hand function in people with high level tetraplegia. Arch Phys Med Rehabil. 2014;95(6):1201–1211.
17. Bouton CE, Shaikhouni A, Annetta NV, et al. Restoring cortical control of functional movement in a human with quadriplegia. Nature. 2016;533:247–250.
18. Donati AR, Shokur S, Morya E, et al. Long-term training with a brain-machine interface-based gait protocol induces partial neurological recovery in paraplegic patients. Scientific Reports. 2016. 6 p.
19. Nicolelis M, Servick K. Interview. Kickoff looms for demo of brain-controlled machine. Science. 2014;344(6188):1069–1070.
20. Aach M, Cruciger O, Sczesny Kaiser M, et al. Voluntary driven exoskeleton as a new tool for rehabilitation in chronic spinal cord injury: a pilot study. Spine J. 2014;14(12):2847–2853.
21. Raspopovic S, Capogrosso M, Petrini FM, et al. Restoring natural sensory feedback in real-time bidirectional hand prostheses. Science Translational Medicine. 2014;6(222):222ra19.
22. Boretius T, Badia J, Pascual Font A, et al. A transverse intrafascicular multichannel electrode (TIME) to interface with the peripheral nerve. Biosens Bioelectron. 2010;26(1):62–69.
23. Guneysu A, Akin HL. An SSVEP based BCI to control a humanoid robot by using portable EEG device. Conf Proc IEEE Eng Med Biol Soc. 2013;2013:6905–6908.
24. Zhao J, Li W, Li M. Comparative Study of SSVEP- and P300-based models for the telepresence control of humanoid robots. PLoS One. 2015;10(11):e0142168.
25. Xie J, Xu G, Wang J, et al. Steady-state motion visual evoked potentials produced by oscillating Newton's rings: implications for brain-computer interfaces. PLoS One. 2012;7(6):e39707.
26. Xie J, Xu G, Wang J, et al. Addition of visual noise boosts evoked potential-based brain-computer interface. Scientific Reports. 2014. 4 p.
27. Chen X, Wang Y, Nakanishi M, et al. High-speed spelling with a noninvasive brain-computer interface. Proc Natl Acad Sci U S A. 2015;112(44):E6058–E6067.
28. Azar NR, Kallakuri S, Chen C, et al. Strain and load thresholds for cervical muscle recruitment in response to quasi-static tensile stretch of the caprine C5-C6 facet joint capsule. J Electromyogr Kinesiol. 2009;9(6):e387–e394.
29.  Chen C, Lu Y, Cavanaugh JM, et al. Recording of neural activity from goat cervical facet joint capsule using custom-designed miniature electrodes. Spine (Phila Pa 1976). 2005;30(12):1367–1372.
30. Chen C, Lu Y, Kallakuri S, et al. Distribution of A-delta and C-fiber receptors in the cervical facet joint capsule and their response to stretch. J Bone Joint Surg Am. 2006;88(8):1807–1816.
31. Lu Y, Chen C, Kallakuri S, et al. Neural response of cervical facet joint capsule to stretch: a study of whiplash pain mechanism. Stapp Car Crash J. 2005;49:49–65.
32.  Lu Y, Chen C, Kallakuri S, et al. Neurophysiological and biomechanical characterization of goat cervical facet joint capsules. J Orthop Res. 2005;23(4):779–787.
33. Tigra W, Guiraud D, Andreu D, et al. Exploring selective neural electrical stimulation for upper limb function restoration. Eur J Transl Myol. 2016;26(2):6035.
34. Tyler DJ, Durand DM. Functionally selective peripheral nerve stimulation with a flat interface nerve electrode. IEEE Trans Neural Syst Rehabil Eng. 2002;10(4):294–303.
35. Badia J, Boretius T, Andreu D, et al. Comparative analysis of transverse intrafascicular multichannel, longitudinal intrafascicular and multipolar cuff electrodes for the selective stimulation of nerve fascicles. J Neural Eng. 2011;8(3):036023.
36. Akin T, Najafi K, Smoke RH, et al. A micromachined silicon sieve electrode for nerve regeneration applications. IEEE Trans Biomed Eng. 1994;41(4):305–313.
37. Hong N, Joo S, Nam Y. Characterization of axonal spikes in cultured neuronal networks using microelectrode arrays and microchannel devices. IEEE Trans Biomed Eng. 2016.
38. Sando IC, Leach MK, Woo SL, et al. Regenerative peripheral nerve interface for prostheses control: electrode comparison. J Reconstr Microsurg. 2016;32(3):194–199.
39. Bareket Keren L, Hanein Y. Carbon nanotube-based multi electrode arrays for neuronal interfacing: progress and prospects. Front Neural Circuits. 2012;6:122.