Exploration of novel methods to reconstruct ECGs from video data

Electrocardiography (ECG) usually requires a contact device, which is uncomfortable for patients. This thesis explores a novel pipeline that combines different methods for reconstructing ECGs from videos recorded using webcams or smartphone cameras via remote photoplethysmographic (rPPG) signals. We used the LGI-PPGI database and the Pulse Rate Detection (PURE) dataset, containing 1-minute videos of faces during four and six different tasks, respectively. We used three traditional methods to extract rPPG signals from the videos based on an RGB time series. These rPPG signals plus the RGB time series have been used to train a model based on Bidirecional LSTM (BiLSTM) neural network, to obtain a good-quality measure of the rPPG. In parallel, an inter-subject model has been developed to map the PPG into ECG, training it on the MIMIC III database that contains 50 healthy subjects and 50 not healthy. Then, rPPG has been used to feed the inter-subject model in order to measure the remote ECG (rECG). To evaluate the PPG to ECG inter-subject model we used Root Mean Squared Errors (RMSE) and the Pearson’s coefficient (r coefficient), exploring also the importance of different window lengths in the segmentation process and cross-validating among the dataset. Note that the signals in input and output of the models are normalized between 0 and 1, so the results are dimensionless. The different window lengths didn’t show a significally difference, in fact the RMSE is always between 0.1 and 0.2 and the r coefficient between 0.35 and 0.55. To evaluate the rECG obtained, we compared the ECG reconstructed from the rPPG signals (remote ECG) and the ECG reconstructed from the contact PPG (reference ECG or ECG ground truth). We used RMSE between the rECG and the ECG ground truth, the mean absolute error (MAE) between the heart rate (HR) estimated from the rECG and from the PPG (MAE beats per minute [MAE bpm]) in both datasets; the Dynamic Time Wrapping (DTW) value between the rECG and the reference ECG and, finally, the Pearson’s coefficient. The RMSE average ranged between 0.11 and 0.13 for the PURE dataset and between 0.17 and 0.22 for the LGI-PPGI database. The MAE bpm average was 2 bpm at most for all the tasks for PURE dataset, and beetween 6 bpm and 8 bpm for LGI-PPGI database. These results confirm that rECG reconstruction, through rPPG signals, is possible and that, for this approach, the rPPG has to be improved in order to improve also the rECG measurement.