Facial expression analysis for automatic detection of cognitive impairment

This thesis contributes to dementia research by employing machine learning techniques, specifically focusing on facial emotion recognition to investigate emotional states as possible early markers of dementia. Dementia is one of the major causes of disability and dependency among older people worldwide and is the seventh leading cause of mortality on a global scale. Receiving a timely diagnosis enables patients and caregivers to enhance their quality of life and get the opportunity to participate in clinical trials. For an early detection of dementia disease, a precursor stage of dementia known as Mild Cognitive Impairment is investigated. In this phase, the first symptoms occur, as memory loss, trouble concentrating, disorientation, communication issues, and changes in mood and behaviour, but the daily activities are still not compromised. In literature it has been observed that cognitive impaired and healthy subjects exhibit different emotional reactions to target emotional stimuli, with an increase in negative emotions for cognitive impaired subjects. Since facial expressions are forms of non-verbal communication providing hints for human emotions, in this research work facial expression analysis is conducted to study if emotional responses could be used to identify possible early markers of dementia. An Artificial Intelligence system is trained on video data collected from cognitive impaired (including Mild Cognitive Impairment and dementia) and healthy subjects, following an emotional elicitation protocol with image and audio stimuli. The first part of the system includes an emotion recognition model. Several Convolutional Neural Network architectures are trained and compared on the AffectNet dataset on the emotion recognition task using the dimensional model of emotions, i.e., extracting arousal and valence values from an image. Valence indicates how much an emotion is positive or negative, while arousal measures its intensity. Once the most suitable emotion recognition model is chosen, it is used to extract the emotional state from the videos of the collected dataset, in terms of time series of valence and arousal values. These data are subsequently used to train a classifier to distinguish cognitive impaired individuals from healthy subjects. The study explores how different variables of the system impact on the detection of cognitive impairment, as camera recording settings and Neural Network parameters. Moreover, the results suggest that facial expressions could serve as early indicators of cognitive impairment.