Integration and Validation of an Event-driven sEMG-based Embedded Prototype for Real-time Facial Expression Recognition

Facial Expression Recognition has demonstrated significant potential in biomedical research area: evaluation of emotional well-being, support in non-verbal communication, control of Human-Machine Interfaces (HMI) and assistance in rehabilitative procedures. While computer vision is currently the dominant approach for facial expression recognition, recent research has shown increasing surface ElectroMyoGraphy (sEMG) use. sEMG is a non-invasive technique to acquire the electrical signals generated by skeletal muscles during contraction by applying non-invasive electrodes on the skin. Several parameters can be extracted from sEMG signals, obtaining accurate and direct measures of muscle activity suitable to digital processing, e.g. machine learning (ML). This thesis presents an implementation of a low-power prototype for facial expression recognition based on the Averaged Threshold Crossing (ATC) technique applied to facial sEMG signals. ATC is an event-driven feature extraction technique in which an event is generated every time the sEMG signal exceeds a threshold. The ATC value is the average of the TC events over a time window (i.e., 130 ms) and has been proven to be an index for muscle activation level. The event-driven approach significantly reduces the amount of data contained in sEMG signals and the complexity of the processing, thus making it well-suited for power-constrained and real-time applications. The central processing unit is an Apollo3 Blue Micro-Controller Unit (MCU) with an ARM Cortex-M4 processor. Five acquisition boards acquire and filter the sEMG signals to extract the ATC. A Bluetooth Low Power (BLE) module Integrated into the MCU and a USB-dongle connected to an external device provide BLE connectivity for wireless data transmission. A control software handles all data streams and has a Graphical User Interface (GUI) on its highest layer. The GUI eases user control during the development, training, and testing phases. Preliminary tests were conducted to establish the expressions to be recognized, the corresponding muscles and the appropriate electrode positioning, integrating the information from previous works. For the dataset creation, 24 healthy subjects were recruited. Each participant performed a list of 11 gestures for 6 repeated sessions. As an ML classification algorithm, a fully connected Artificial Neural network (ANN) was chosen to fit the MCU limited hardware resources. This is the most straightforward ANN architecture and can be easily implemented with optimized ARM libraries. The prediction process only uses matrix multiplication requiring minimal computational overhead. Different ANNs were trained and validated using k-fold cross-validation. The best performing ANN (1 hidden layer, 42 nodes) was then exported to the MCU for testing. The system was tested on 6 additional healthy subjects reaching 97.4 % average accuracy. During active prediction functioning, power consumption measurements on the MCU showed a 0.582 mA mean current absorption. The average prediction latency of the classifier was measured 0.627 ms . The maximum application latency is 205.627 ms, considering the ATC window contribution (130 ms), the maximum BLE connection interval (75 ms) and the prediction latency (0.627 ms). The low current absorption, the application latency (<300 ms) and the classification accuracy shown by this approach meet the requirements for a future realization of a wearable system for real-time facial expression recognition.