Machine Learning and data fusion of physiological signals for assessing a subject’s stress level and cognitive load

The increase in artificial intelligence integration across various industries underlines its significant impact on promoting secure working spaces, especially in preventing accidents, and mitigating risks. With human-machine interaction (HMI), operators are often tasked with performing complex duties, raising mental workload and stress levels, potentially compromising their performance, and elevating the risk of accidents. Stress refers to the psychological and physiological responses elicited by perceived demands exceeding an individual's coping capacity, while cognitive workload denotes the amount of mental effort and resources required to perform tasks effectively. Existing literature shows a correlation between these cognitive states and the physiological signals of the human body. To investigate this relationship further, the eLions group at Politecnico di Torino conducted a study involving 61 subjects. These participants underwent “n-back” and “Stroop” tests to induce cognitive load and stress state alteration, respectively. Their physiological data was monitored and assessed throughout the tests. N-back tests are conducted with visual, auditory, or combined stimulation, with three levels of difficulty, while the Stroop test had only one level. In total, four datasets were obtained. This study aimed to develop an innovative model that classifies stress and cognitive load levels based on machine learning algorithms using multimodal physiological signals obtained previously. To achieve the specified goal, an analysis of the state of the art was conducted as a starting point to find procedures suitable for the task. Initially, models were developed for the binary classification of samples into two states: rest and altered state. In all four datasets, results of 100% accuracy were achieved. Afterward, they were developed for the multiclass classification into four states: rest and three levels of cognitive load or stress. In analyzing the Stroop dataset, a classification accuracy of 67% was attained for stress classification through a combination of LDA as dimensionality reduction and then a Support Vector Machine as the classifier. Meanwhile, cognitive workload classification for the Visual N-back task yielded an accuracy of 80% using K-PCA with LDA, while for the Audio N-back task, it was 65% using PCA and LDA. Additionally, the Dual N-back task achieved an accuracy of 82% with the blend of LDA and SVM. In addressing the challenge of heavily imbalanced classes in the Stroop test and Dual N-back dataset, outliers were identified and removed, leading to a transition to a 3-class classification. As a result, 75% and 85% accuracies were achieved for the Stroop test and Dual N-back tasks. Moreover, models have been created to evaluate scenarios without one or more signals involved. The project has resulted in the development of a comprehensive and versatile framework able to retrieve, manipulate, and accurately classify the differentiation between a state of rest and an altered state based on an individual's physiological parameters. Furthermore, it can discern various degrees of cognitive workload and stress severity, even when certain biological signals are absent. This framework lays the groundwork for creating a safety device that analyses physiological signals to assess the operator's condition, especially during critical moments. It could enhance safety by providing real-time evaluations of the operator's state, thereby reducing potential risks and ensuring safer operations.