polito.it
Politecnico di Torino (logo)

Understanding and Recognizing Facial Expressions based on Deep Learning

Renju Wang

Understanding and Recognizing Facial Expressions based on Deep Learning.

Rel. Federica Marcolin, Francesca Nonis. Politecnico di Torino, Corso di laurea magistrale in Mechatronic Engineering (Ingegneria Meccatronica), 2024

Abstract:

Facial expressions stand out as one of the most powerful non-verbal ways to express human emotions and intentions. These expressions are often deliberate and socially regulated, making them more straightforward to analyze and understand. In addition to these ordinary facial expressions, known as macro-expressions, that we observe daily, there are also spontaneous facial expressions, especially micro-expressions, are involuntary, subtle and occur briefly, often without the awareness of the individual displaying them. These unique characteristics make micro-expressions challenge to detect. To better understand the rapid and involuntary nature of micro-expression, we utilize the optical flow algorithm to calculate facial movements between the onset and apex frame of the micro-expression videos. This method provides a dynamic map of facial movements and the visualization of the subtle facial movement. The analysis of facial expressions has been a long-standing research focus in the field of computer vision, significantly advanced by applications in human-computer interaction. Early methods for facial expression recognition mainly relied on traditional manual feature-based techniques and mapped facial expressions to basic emotion categories. For better understand the complexity of emotions, we project facial expressions into the two-dimension valence-arousal space. We utilized public BU3DFE dataset and CalD3r datasets. Reorganized the data labels based on the discrete emotional categories and the valence-arousal dimension. With the recent success of Deep Learning across numerous tasks, neural networks have gained increasing attention for their potential in spontaneous expression recognition. We utilized pre-trained deep learning models like VGG16, MobileNetV2 and ResNet50,all of which were originally trained on large ImageNet datasets. By applying transfer learning techniques and fine-tuning these models on our small datasets, we effectively enhanced feature extraction and improved expression recognition accuracy. To enhance the robustness of the model, we performed the data augmentation, which can generate more training samples. We use the accuracy measurement like precision, recall and F1 score to evaluate the model’s performance. Additionally, visualization techniques such as Grad-Cam and optical flow were employed to illustrate and compare the dynamics of facial movements, thereby enriching our understanding and the capability to detect and analyze facial expressions.

Relators: Federica Marcolin, Francesca Nonis
Academic year: 2023/24
Publication type: Electronic
Number of Pages: 70
Additional Information: Tesi secretata. Fulltext non presente
Subjects:
Corso di laurea: Corso di laurea magistrale in Mechatronic Engineering (Ingegneria Meccatronica)
Classe di laurea: New organization > Master science > LM-25 - AUTOMATION ENGINEERING
Aziende collaboratrici: Politecnico di Torino
URI: http://webthesis.biblio.polito.it/id/eprint/32088
Modify record (reserved for operators) Modify record (reserved for operators)