Multi-modal EMG-based hand gesture classification for the control of a robotic prosthetic hand

Abstract

Upper-limb myoelectric prosthesis control utilises electromyography (EMG) signals as input and applies statistical and machine learning techniques to intuitively identify the user’s intended grasp. Surface EMG signals recorded with electrodes attached on the user’s skin have been successfully used for prostheses control in controlled lab conditions for decades. However, due to the stochastic and non-stationary nature of the EMG signal, clinical use of pattern recognition myoelectric control in everyday life conditions is limited. This thesis performs an extensive literature review presenting the main causes of the drift of EMG signals over time, ways of detecting such drifts and possible techniques to counteract for their effects in the application of upper limb prostheses. Three approaches are investigated to provide more robust classification performance under conditions of EMG signal drift; improving the classifier, in corporating extra sensory modalities and utilising transfer learning techniques to improve between-subjects classification performance. Linear Discriminant Analysis (LDA), is the baseline algorithm in myoelectric grasp classification applications, providing good performance with low computational requirements. However, it assumes Gaussian distribution and shared co-variance between different classes, and its performance relies on hand-engineered features. Deep Neural Networks (DNNs) have the advantage of learning the features while training the classifier. In this thesis two deep learning models have been successfully implemented for the grasp classification of EMG signals achieving better performance than the baseline LDA algorithm. Moreover, deep neural networks provide an easy basis for transfer learning knowledge and improving the adaptation capabilities of the classifier. An adaptation approach is suggested and tested on the inter-subject classification task, demonstrating better performance when utilising pre-trained neural networks. Finally research has suggested that adding extra sensory modalities along EMG, like Inertial Measurement Unit (IMU) data, improves the classification performance of a classifier in comparison to utilising only EMG data for training. In this thesis ways of incorporating different sensory modalities have been suggested, both for the LDA classifier and the DNNs, demonstrating the benefit of multi-modal grasp classifier.