Real-time algorithm for heart rate extraction from in-vehicle remote photoplethysmography

The rise of photoplethysmography (PPG) in recent years has brought non-invasive cardiovascular monitoring into the spotlight, as it offers a unique approach to heart rate assessment by capturing blood volume variations over time. Furthermore, when it is preferable to avoid direct contact between the instrument and the subject, remote or non-contact PPG (rPPG) can be employed. This technique utilizes acquisition systems that eliminate the need for physical contact, such as cameras, to extract the rPPG signal by averaging pixel intensity over time. Not only has this technology been adopted in clinical settings, but also in other environments where monitoring vital parameters can offer advantages. One such area with great potential is in-vehicle monitoring. Driving is inherently risky, especially when stress, drowsiness, and fatigue affect a driver's performance, thus potentially leading to accidents and endangering people on the road. Remote PPG employs optical sensors to non-invasively monitor vital signs such as heart rate, facilitating early detection of physiological changes and alerting the driver and vehicle systems in real-time. In addition, this camera-based technique can also be integrated with other technologies to capture data on the driver's facial features, allowing a more complete understanding of the driver's physical state. This thesis project focuses on the development of a real-time algorithm for extracting heart rate from rPPG signals, with the goal of future in-vehicle applications. The acquisition system is designed and produced by STMicroelectronics, and it consists of a global-shutter monochromatic camera with a near-infrared optical filter and two LEDs emitting at a wavelength of 950 nm . The first phase of the algorithm consists in extracting the rPPG signal from specific regions of interest (ROIs) on the subject's face. The face and ROI detection is performed by MediaPipe Face Mesh, a Machine Learning-based algorithm provided by the MediaPipe framework. Since the signal is significantly affected by noise from instrumentation, micro and macro movements of the body, and lighting conditions, a dual filtering process involving a bandpass filter and a Kalman filter is implemented. Once the signal has been cleaned and the PPG waveform is visible, peak detection and heart rate calculation come as the next step. Further corrections of potentially inaccurate heart rate results are made by analysing the signal's historical data: this corrective approach contributes to the improvement of final heart rate calculation performance. Finally, a Machine Learning classifier has been built to assess the quality and reliability of the signals from different ROIs, with the aim of providing the most effective combination of signals and therefore the most accurate heart rate results. The developed system produces reliable real-time heart rate values, specifically when the subject remains relatively motionless, despite the inherent complexity of the problem. Future steps include improving the Machine Learning-based reliability evaluator by expanding the dataset, and the introduction of a motion artefact reduction system. These advances will be crucial for adapting the system to in-vehicle applications, ensuring accurate and robust heart rate monitoring even in dynamic driving environments.