Emotion Recognition: a study case on identifying VR induced emotional states by using EEG signals.

The scientific knowledge related to emotions has always been limited. In literature it has been already proven the correlation between the emotional indexes of Valence, Arousal and Dominance, which are functions of EEG signals θ, α, β, γ, and emotions. However, in the last years, the research has shown an increasing interest in establishing “emotional” connections between humans and computers and the need for developing reliable and deployable solutions for the detection and evaluation of human emotional states. In fact, recent research has shown how consumer wearable EEG could be used in identifying emotions. In this context, this project aims at understanding if and how the electroencephalography (EEG) can be used to identify emotions. To do this, an experiment was conducted at the 3D Lab of the Politecnico di Torino. Twenty-seven participants were preliminarily required to perform an empathy and alexithymia test to define their emotional profiles. Then they were asked to navigate through ten Virtual Reality scenarios, divided in five open and five closed environments. These scenarios were created to elicit a certain emotion in the people visiting them: two scenarios (one open and one closed) for happiness, two for disgust, two for fear, two for anger, two for sadness. While the user was navigating the environments, he/she was wearing the wearable EEG headset Emotiv EPOC+ for EEG signals analysis and he/she was filmed by a sensor IntelRealSense SR300, belonging to the family of RGB-D cameras, to perform Facial Expression Recognition (FER). At the end of each scenario’s visit, the user was asked to answer a questionnaire and to classify Valence, Arousal and Dominance and happiness, disgust, fear, anger, sadness to label the emotion they predominantly felt. In this way, the MODA of Valence, Arousal and Dominance were associated to the Self-Assessment Manikin (SAM) images and compared to understand the emotional changes in every environment, while the emotions classification was needed to prove that the scenarios elicit the emotion they were designed for. This work aimed at identifying a visual scheme to guide readers and researchers to the comprehension of EEG-based Emotion Recognition. For this purpose, Valence, Arousal and Dominance were computed through their mathematical relations with the EEG signals, which were captured during the EEG analysis and then their temporal behaviour was represented. The activation of the Emotiv EPOC+ sensors was represented in a dedicate graph and then used to identify the most active cerebral area for every scenario and every brain wave. Last, for each scenario the SAM images of Valence, Arousal and Dominance obtained from the EEG data and the FER data were compared and were used as indicators of the emotional state that the users felt while navigating the environments. All these representations were gathered to form the “visual scheme” for every VR scenario to be used for the emotion recognitions via EEG signals. EEG-based Emotion Recognition is crucial for the development of machines to be integrated for the benefit and application in society. In fact, it could be helpful in education to observe students’ mental state towards teaching materials, or in medicine to allow doctors to assess their patients’ mental state and to get better treatments for their psychological conditions, or more generally to understand a worker mental conditions during normal and stressful situations.