Energy-Efficient Deep Learning-based Heart-Rate Estimation on Wearables

Nowadays, Deep Learning (DL) is predominant in many fields like Computer Vision or Natural Language Processing, with state-of-the-art and sometimes super-human accuracy. On the other hand, deploying a network in a real-world embedded system still poses several challenges. First, the data collected is often corrupted and can hinder a correct prediction of the network. Second, Deep Neural Networks (DNNs) are usually too big to fit the tight constraints of an embedded platform (for instance, a limited memory of few MBs) and need manual tuning to be optimized and reduce their dimensions while still achieving good accuracy. Integrating AI predictions directly on edge devices like wearables can be really helpful in many situations where the real-time monitoring of the user is needed. For instance, Heart-Rate (HR) monitoring is becoming increasingly more linked to the analysis of PhotoPlethysmoGraphic (PPG) signals, which can be extracted from wrist-worn devices. Such a technique is portable, cheap and comfortable, compared to the previously dominant one based on Electrocardiogram (ECG), which is more intrusive and whose collection impairs the daily life of the subject. However, the benefits of the PPG-based HR monitoring do not come without downsides. The main problem of PPG is the presence of Motion Artifacts (MA) generated by the movements (especially in wrist-worn wearables) and the infiltration of light between the skin and the sensor, that create noise in the collected signal. The focus of this work is the development of a complete system able to collect the PPG and track the HR of the subject while removing MAs. The system consists of i) a Client-Server interaction based on the Bluetooth LE communication protocol to send PPG data / HR estimation to a collecting device (e.g., a smartphone), ii) a deep neural network model (specifically a Temporal Convolutional Network -- TCN) optimized for the execution on the edge, and iii) a simple controller based on a Finite State Machine (FSM) that manages the collection of data, the prediction of the HR and the transmission of data. The selected edge devices are the STM32WB55 Nucleo development board, for the prototyping phase, and a real wearable device called H-Watch, developed by ETH Zurich for the final deployment. Specifically, both devices feature 1 MB of Flash Memory and a STM32WB55RGV6 SoC (System on Chip) by ST Microelectronics with two independent cores: an ARM® Cortex™-M4 at 64MHz and ARM® Cortex™-M0+ at 32MHz, dedicated to the Bluetooth Low Energy (BLE) stack. Additionally, the H-Watch comes with different sensors, among which a pulse oximetry (MAX30101) and 6-axes IMU (LSM6DSM), allowing the collection of data. The dataset that we used to benchmark our results is PPGDalia, the largest publicly available collection of PPG and tri-axial accelerometer data obtained during daily life activities. The neural network chosen to be deployed is a Temporal Convolutional Network, known to work well with time series-like data, such as the stream of PPG signals over time. The model is optimized for edge execution, and it consists of around 97K parameters, occupying 374.75 KB of the total 1MB memory space in the MCU. On the PPGDalia dataset, the network running on the STM32WB55 achieves a Mean Absolute Error (MAE) of 2.36 Beats per Minute (BPM) compared to the golden HR computed with an ECG band.