Can Virtual Reality evoke emotions? A case study on emotional expressions induced by ten newly developed affective environments

Emotions can be defined as an interwoven process, constituted by numerous elements. The means through which we express emotions and identify those belonging to others is given by the face. Emotions’ facial expression is a subject of researches, particularly for those scholars claiming that automatic facial expressions recognition and analysis have been of central focus in fields concerning Artificial Intelligence, Affective Computing and Human-Computer Interaction (HCI). Those researches are even of specific interest in the following areas: neuroscience, clinical psychology, informatics, marketing and security. The main purpose of the experiment, part of an extensive project, carried out in order to study ecologically valid emotional expressions evoked by the visualization of some virtual environments in the laboratory. During this experiment subjects have observed 13 scenarios in Virtual Reality (that means, planned triggers expressly validated by the team) that evoked emotions in the participants. 31 subjects, whose age ranges from 19 to 34 years old, have been recruited and of these, 4 were Pilot-subjects, who were excluded from the data analysis because the experimental conditions were not similar to those in the sample. All the participants, however, have received in advance questionnaires about alexithymia and empathy. The choice of these questionnaires has been made to guarantee the presence of the metacognitive requirements concerning the users engaged in this project. After that, subjects visualize virtual scenarios, and, simultaneously, their faces are acquired through the Intel RealSense SR300 sensor to obtain both 2D (RGB) and 3D (Depth) frames. Following the scenarios’ visualizations, participants have judged the emotions they have felt through some self-evaluation questionnaires (the SAM, the semantic scale of emotions and the post-test surveys). The facial expression resulting from the emotions previously felt has been captured in video frames, thus obtaining a faces’ database. Expressions have been classified by a focus group and later sent as input to an algorithm of deep learning, namely a Convolutional Neural Network (CNN), trained to categorize automatically neutrality and the six Ekman’s emotions (that is anger, disgust, fear, happiness, sadness, surprise). Once data have been collected, they have been elaborated to prove if virtual scenarios are functional at inducing spontaneous emotions instead of affective static pictures used in the parallel experiment; and to demonstrate the existence of a correlation between the emotions declared by the participants, those facially expressed and those classified by the network. Even though we did the experimentation by including the Surprise’ scenarios, at the end we analysed just the newly designed environments since the first ones had been developed to elicit Awe, not really a Surprise. Among the 11 scenarios taken into account, those which have been more efficient have been the ones concerning Happiness-Outdoor, Disgust-Outdoor and Anger-Outdoor. Moreover, by examining the network’s classification it has been noticed a higher accuracy with respect to the recognition of Happiness, although a disequilibrium of the classification towards the Neutrality had taken place: this allows to suppose that many frames categorized, depending on the six emotions, present an extremely low level of activation, reason for which the barely marked expression can be linked to neutrality.