Face and gait fusion techniques for vehicle owner recognition

Intelligent vehicle applications are transforming the landscape of automotive technology, offering advancements that enhance safety, security, and user experience. A critical aspect of these applications is the ability to accurately identify and authenticate the owner of the vehicle, ensuring that only authorized users can operate or interact with the vehicle's systems. The research developed in this thesis work is part of a larger initiative by Centro Ricerche Fiat (CRF) and Stellantis, which aims to develop cutting-edge security mechanisms for intelligent vehicles. Specifically, the study investigates the combination of face recognition and body pose estimation leveraging gait analysis to create a multimodal authentication system able to achieve robust and secure human authentication while approaching the vehicle from the outside. This work specifically focuses on evaluating various advanced fusion techniques, including polynomial feature fusion, hierarchical feature fusion, approaches based on deep learning architectures and other approaches, comparing them with state-of-the-art results provided by previous studies involving score level fusion techniques. Preliminary results, have shown promising outcomes, indicating that the combination of face and gait recognition can significantly enhance the security of intelligent vehicle systems. This work aims to build on such findings by focusing on feature level fusion, which is expected to provide a more detailed and accurate representation of the biometric data. Polynomial feature fusion aims to capture nonlinear interactions between the features of different modalities, potentially leading to more robust decision-making. Hierarchical feature fusion introduces a multi-layered process where features from both modalities are fused at different stages, offering a deeper and more structured integration. On the other hand deep learning-based fusion techniques offer the advantage of learning complex feature representations and interactions from large amounts of data without requiring hand-engineered feature extraction rules. The investigation involved an extensive experimental comparison of these fusion techniques, analyzing their respective benefits and limitations. The dataset employed for this study is CASIA-B, a comprehensive database widely used for gait recognition research. In addition to the feature fusion techniques, this work also explores the impact of image upscaling on biometric recognition accuracy, both for score-level and feature-level fusion. Image quality plays a crucial role in biometric systems, especially in the case of facial recognition, where higher-resolution images can potentially lead to better feature extraction and matching performance but unfortunately also to significant computational costs. The research highlighted the existing limitation both from a computational point of view and from the low number of research available for this area and propose potential improvements for the future.