TIMING OF FACIAL EXPRESSIONS: A FOCUS ON MICRO-EXPRESSIONS AND SELF-SOOTHING MECHANISMS

The primary objective of this thesis’s project is a component of a comprehensive work designed to investigate emotionally authentic expressions triggered by the presentation of specific virtual environments in a controlled laboratory setting. In this research, the focus is on examining the duration of facial expressions concerning the elicited emotion and classifying these expressions into the three primary categories: macroexpressions, microexpressions and self-soothing expressions. In this experiment, participants were exposed to 13 scenarios in Virtual Reality, carefully designed to elicit emotional responses. A total of 31 participants, aged between 19 and 34, were recruited for this study. Prior to their participation, all individuals completed questionnaires assessing alexithymia and empathy. The selection of these questionnaires was deliberate, ensuring the presence of necessary metacognitive aspects in the participants engaged in this project. During the experiment, participants were immersed in virtual scenarios while their facial expressions were simultaneously recorded using the Intel RealSense SR300 sensor, capturing both 2D (RGB) and 3D (Depth) frames. Following the exposure to these scenarios, participants were asked to assess the emotions they experienced through self-evaluation questionnaires, which included the SAM, the semantic scale of emotions, and post-test surveys. The facial expressions reflecting the previously experienced emotions were recorded in video frames, creating a comprehensive database of facial responses. The recorded facial expressions were subjected to a meticulous analysis aimed at identifying macroexpressions, microexpressions and self-comforting mechanisms. Following this categorization, the videos falling into the first two expression categories were fed into a deep learning algorithm known as the Convolutional Neural Network (CNN). This CNN was employed to classify these expressions into Paul Ekman's basic emotions, including videos displaying no discernible expression (representing a neutral state). On the other hand, the self-soothing expressions were subjected to a different machine learning algorithm known as k-nearest neighbors (K-NN). This algorithm was employed for the classification of self-soothing expressions. The outcomes of this study reveal that facial expressions exhibit varying durations contingent on the emotions evoked by the virtual reality scenarios. Notably, happiness, in addition to being one of the most frequently occurring emotions, manifests as the one with the longest duration. Furthermore, it was observed that during stressful situations participants displayed self-soothing mechanisms and microexpressions, with the latter having a shorter duration compared to macroexpressions. In the end, the two implemented algorithms achieved a high level of accuracy in classifying macroexpressions and self-soothing expressions. However, their performance was less satisfactory in identifying microexpressions due to the limited number of available videos. Microexpressions, characterized by distinct durations and intensities in comparison to macroexpressions, presented challenges for the network's accurate recognition.