Automatic analysis of experimental data for facial expression recognition in an ecologically valid database creation

Facial expressions have a universal communicative value and are naturally recognized by human eye. There are several application’s sectors, spanning from welfare and security fields to neuromarketing, having the purpose to obtain an automatic process of Facial Emotion Recognition (FER) through human expression analysis. In this dissertation it’s analysed this ability, conveyed to an Artificial Intelligence, in order to find its accuracy. Indeed, the capability to acquire the automatic FER comes from Machine Learning, a method through which an algorithm is educated with the aim of gaining experience. Once this training process is finished, the Convolutional Neural Network (CNN) becomes expert and ready to be applied to general cases. The CNN employed is a particular architecture of Deep Learning, due to its complex structure of 150 hidden layers, whose goal is to find and learn the image features. The ML efficacy is linked to the data used for training the specific algorithms. The experimental database of this thesis is obtained from 35 volunteers, aged from 18 to 35. Alexithymia and Empathy tests have been submitted to each one of them to comprehend the experiment suitability. Subjects’ emotions are elicited by the vision of 54 images, organized in 2 phases: training and testing, composed by 6 and 48 pictures respectively; these ones have been taken from the IAPS (International Affective Picture System) and the GAPED (Geneva Affective Picture Database) combination, which are databases conceived to arouse emotions. During the image visualization, the expression of the user is recorded and, after each display, the subject self-assesses the elicited feelings through the SAM scale (Self-Assessment Manikin), a technique of affective non-verbal rating, which employs 3 different scales: valence, arousal and dominance. The volunteers’ familiarization with the SAM and the image stimuli typology occurs during the training phase. Furthermore, it’s asked to indicate the emotion felt among the Ekman’s 7 principal ones: anger, fear, disgust, happiness, sadness, surprise and neutral. These self-evaluations are needed for the final comparison of results. The experiment takes place in the 3D Lab of Polytechnic of Turin. The video sequence is recorded by an RGB camera and an Intel Real Sense RS300 sensor that employs a structured light depth camera, providing a 3D map of the acquisition. From the recorded videos, using the RealSense Viewer software, the frame with the most significative expression is selected, then RGB and Depth map videos are aligned and cropped, in order to be used for the database employed for training and testing the CNN. The FER can be equivocal also for a human observer; hence, the emotions’ choice has been taken by an interdisciplinary team group (including a psychologist) to make it the most accurate as possible. Particular attention is given to the comparison among the elicited emotion, the one supposed to be aroused by the image display and the one the subject seem to express visually according to two points of view: human eye and neural network. The evaluation of results takes also into consideration the Alexithymia and the Empathy tests of participants. Next step will be to enhance the database with a more heterogeneous sample. In fact, this research provides an experimental basis for further improvements with additional complexity, with the goal of obtaining a CNN for real life contexts and different application fields.