Comparative Analysis of Computer Vision Approaches for Facial Expression Recognition

The way people communicate is closely tied to the expression of their emotions. Facial expression analysis has become a hot study issue in recent years, with applications in a variety of fields, including the Human-Computer Interaction (HIC). Thanks to Computer Vision technologies, Facial Expression Recognition (FER) uses algorithms to analyze faces in images or video, integrating the visual world with computing systems through the simulation of human vision. FER employs facial feature and movement analysis to recognise emotional expressions. FER analysis consists of three steps: Face Detection, Facial Expression Detection, and Expression Classification into emotional states. There are many reasons why interpreting emotions from Facial Expressions may not always be accurate as mix different emotions at the same time or sociocultural and contextual factors. Its quality can also be affected by technical elements, including illumination and varied camera angles. At the moment, State of the Art on FER software can attain an Accuracy of between 75 and 80 percent. It is interesting to contrast this with the average natural human capacity to identify emotions, which is approximately 90%. Moreover, FER algorithms frequently encounter compatibility issues with devices’ processing capability. Among the most widely used databases for FER systems are the BU-3DFE and the Bosphorus, thus they have been utilized during the development of this work. Following Ekman’s theory on Basic Emotions, they provide seven classes of expressions: Anger, Disgust, Fear, Happiness, and Surprise are the Basic Emotions, since they are recognized as universal emotions, and Neutral expression, which is defined as no emotion. Three different approaches have been carried out to classify the facial expressions. For the first approach, Convolutional Neural Networks (CNNs) were used for automatically extracting features and classifying expressions in FER systems. It was chosen to assess different networks separately for each of the seven classes. Five pre-trained models from the Tensorflow Keras module have been compared: VGG16, ResNet50, Xception, MobileNet and EfficientNet. It has been opted for Transfer Learning and fine-tuning method since using the weights from the pre-trained models on one task to initialize the weights of the new model for a new task can be advantageous for Accuracy and computation cost. In order to improve the Accuracy, for the second approach, it has been proposed to reduce the number of classes that the classifiers must distinguish between by grouping them according to shared attributes, generating a cascade of classifiers. These approaches provide good results among the State of the Art, but the computational cost is quite high to be used with underperforming devices. Thus, for the third approach, the CNNs have been employed just to extract features from the images. Afterward, the PCA method has been used to select the most important features, and a Machine Learning (ML) classifier, called Support Vector Machine (SVM), was selected to classify the expressions. The evaluations highlight that combining different methods based on Machine Learning and Deep Learning (DL) algorithms can be very advantageous since it allows to find a compromise between a high Accuracy and a low computational cost. It is possible to believe that, in future work, integrating similar strategies with embedded systems can have a significant impact to the field of facial expression analysis.