AI-Powered Eye Signal Analysis for Real-Time Stress and Mental Workload Monitoring

The relationship between stress, mental workload (MWL), and eye movement patterns is well-established but remains complex and not fully understood. Recently, artificial intelligence (AI) has shown considerable potential across various fields, emerging as a powerful tool for analyzing complex relationships. This thesis builds on previous research that used AI to assign a single class per task in postprocessing based on six biological signals. Here, the objective is to achieve real-time classification using only eye movement data, assigning a stress or MWL class every few seconds. Conducted at Politecnico di Torino, this study involved 103 participants in an experimental design aimed at inducing stress and MWL through two cognitive tasks: the Stroop test to induce stress and the N-Back test to increase MWL, with rest phases interspersed. Each task included three phases of escalating difficulty, with physiological data continuously collected. For each phase, participants rated their perceived stress and MWL levels on a scale from 1 to 3, while rest phases were assigned to 0. Three types of eye movement data were recorded per eye: horizontal and vertical gaze coordinates and pupil diameter. This data was segmented into buffers to simulate real-time conditions, and features were extracted from each segment. Four datasets were created: the Stroop dataset, the N-Back Audio dataset (audio stimuli), the N-Back Visual dataset (visual stimuli), and the N-Back Dual dataset (both stimuli). Binary classification first distinguished stressed from non-stressed participants, multiclass classification then differentiated between varying levels of stress and MWL, with a final three-class classification trained by excluding the least frequent class in the Stroop dataset. Using a fixed overlap for buffer extraction initially led to phase imbalance, with the rest phase dominating. To resolve this, an adaptive overlap algorithm that adjusted based on phase length was tried; however, phase correlation changes resulted in poor classifier performance. Then a custom extraction algorithm was developed, balancing the dataset by identifying the shortest phase and matching buffer counts in longer phases, with buffers extracted from the start, middle, or end of each phase. This method proved most effective and was used in the study. After testing various buffer widths, overlaps, and extraction positions across datasets, the optimal parameter combination was chosen. Binary classification performed well, with F1 scores consistently exceeding 90% across all datasets, including a peak of 99% for the Stroop test. These results surpass the current state of the art for MWL and stress detection, highlighting the effectiveness of using only eye movement data. Multiclass classification results varied by dataset: the Stroop dataset initially reached 69% F1, increasing to 78% upon removing class 3. The N-Back Audio dataset achieved a 60% F1, likely due to the limited alignment of audio stimuli with eye movement characteristics, while the N-Back Visual dataset scored 67% F1. The N-Back Dual dataset yielded the highest F1 at 81%, comparable to the state of the art in real-time stress and MWL classification. Overall, the results match the performance of previous work, but with real-time classification based solely on a single eye movement signal. This method enables faster data processing, allowing stress and MWL classification every few seconds and supporting non-invasive applications in real-world settings.