Automatic 3D facial expression recognition with ecologically valid 3D data

With the advent of Artificial Intelligence (AI) in everyday life, the human-robot interaction is increasingly establishing, as demonstrated for example by the rapid development of different voice personal assistants over the last few years. In this context, the Facial Expression Recognition (FER), is a crucial aspect to design an empathic machine, able to understand the feelings of the facing person and to interact flexibly while considering his emotions. This potentially leads to perform human-like communication that could sensibly improve the human-machine interaction experience. Since the interest in the facial emotion and expression recognition in real settings is spreading faster, it can be determinant having a dataset with human emotions whose manifestation in experimental settings mirrors the phenomena in the real world, rich of various stimuli. The ecological validity of the dataset, created with the 35 subjects summoned in the experiment, in collaboration with the Politecnico di Milano, conducted in the 3D Lab of the Politecnico di Torino, relies on the elicitation of emotions combining two of the most worldwide known affective database, IAPS and GAPED. Furthermore, the interdisciplinary team includes the figure of a psychologist among the engineering ones. The aim of the experiment was realizing and testing an ecologically valid facial expression database by eliciting the main human emotions with a specific affective dataset, without asking the participants to act, but only to react spontaneously to the 48 images administered. The latter have been chosen carefully, considering that the neural network used can distinguish, over the neutral expression, the six basic Paul Ekman’s emotions (i.e. anger, disgust, enjoyment, fear, sadness and surprise). After visualizing the image, each person was asked to rate, in term of arousal and valence through the Self-Assessment Manikin (SAM) scale, the images seen and to label the emotion felt, to better understand if the pictures have yielded the emotion intended, and how the ratings can be distributed in the affective space. Before joining the experimentation, each person was required to compile an empathy and alexithymia test that provides his personality profile. The topic, seeing also the different loaves that intertwine, the scientific, the emotive, the psychological and the technological, offers the opportunity to reflect on important issues, while the neural network processes the facial expressions acquired: how many images have elicited the emotion planned? Is there a correspondence between the expression showed, the emotion labelled, the one expected and personality of the person? Each frame was captured from a sequence recorded simultaneously by an RGB camera and a coded-light depth sensor integrated into an innovative instrument providing a 3D map of the acquisition. Further works could expand the database including people with different proveniences and ages. The efficiency of this eliciting method is discussed and the confusion matrix resulted from the neural network are analysed. Machine-learning techniques could play a key role in FER, considering that the scenario concerning humanized intelligence is wide. Advanced driver assistance systems, robot assistants involved in special needs children therapies, biometric surveillance systems or early detection of emotional impairments, are only few examples of the uncountable and still undiscovered fields of the AI.