Design of a Face and Gait Biometrics Authentication System based on a plug-and-play multi-modal framework

Intelligent and secure access to vehicles is a crucial aspect of the evolving automotive industry, especially in times where the integration of deep learning technologies can enhance both accuracy and efficiency. This research is part of a broader activity in collaboration with Centro Ricerche Fiat (CRF) and Stellantis addressed also through a separate thesis work, which aims to explore ways to redefine user experience and security in situations where conventional access tools, such as keys or phones, may be unavailable. The focus is on developing a system capable of detect, recognize, and verify the identity of vehicle owners based on their biometric traits, while they are walking towards the vehicle in an unconstrained environment (performing so called recognition in the wild). However, traditional authentication systems based solely on Face Recognition (FR) can lead to poor verification results due to variations in lighting, face appearance and occlusion. To cope with this issue, it is proposed to combine information coming from the body (e.g., body pose estimation, body shape, etc.) with information regarding the face, thus strengthening their discriminative power. Among the numerous possibilities, this thesis work specifically explores Gait Recognition (GR) (i.e., recognition of the walking pattern of an individual) as a possible option to enhance verification accuracy in real-world scenarios, driven by the motivation that face and gait may be complementary traits. To this extent, the solution proposed in this thesis consists of a multi-modal framework that covers each step of the identification process, from person detection and tracking, to face and gait recognition. Moreover, the architecture is made of modular “plug-and-play” components, which allow in principle a seamless integration of new models for each one of the steps, without requiring additional training. Leveraging this characteristic, a group of models (denoted as baseline) is selected based on their cost-accuracy tradeoff and provided in this thesis work, whereas alternative model candidates are reviewed in the parallel thesis. Furthermore, to combine face and gait information, two fusion techniques are investigated (score-level fusion and an initial exploration of feature-level fusion), analyzing benefits in terms of recognition and verification accuracy compared to the usage of FR and GR models independently. For the score-level fusion, weighted mean, sum and multiplication of individual scores are employed, whereas feature-level fusion is carried out by normalizing the features for each trait and concatenating them. The presented score-level fusion approach achieves remarkable improvements on the CASIA-B dataset, reporting higher recognition and verification results on most of the experiments when compared to using face and gait models independently. Furthermore, specific configurations of the proposed feature-level fusion yields extremely promising results on both tasks, which makes it a good omen for future developments. Finally, the research provides insights into current limitations and suggests avenues for further improvement, laying a comprehensive foundation for future studies in this field.