Exploring Brain-Inspired Multi-Sensor Data Fusion Models for Improving Performances in Navigation and Tracking Applications

The computational resources necessary for training and utilizing traditional deep learning (DL) models are becoming increasingly costly as time progresses. The amount of computing power required has increased tenfold from 2012 to 2019, leading to a significant rise in costs. Neuromorphic technologies and spiking neural networks (SNNs) are inspired by how human brains work and offer an innovative approach, significantly reducing the required resources. The objective of this thesis is to evaluate the potential for the adoption of brain-inspired solutions as a substitute for established techniques used to solve engineering tasks. In particular, this thesis aims to assess the feasibility of employing SNN-based models to enhance the performance of navigation and tracking tasks. In the first part of the work, we investigate the use of a SNN for implementing a dead reckoning task. We use the Neural Kalman Model, a deep learning-based solution that combines synthetically generated GPS measurements and IMU data, as a baseline. This model uses a Temporal Convolutional Network (TCN) to estimate the parameters of an Extended Kalman Filter state. Our contribution is the replacement of the TCN with a fully spiking variant of the Legendre Memory Unit (LMU), a specialized recurrent cell that captures long-range dependencies using orthogonal Legendre polynomials. The spiking-based solution achieves a significant reduction in memory usage, improving efficiency by nearly 96%, and enhances accuracy, reducing error by 53%. In the second part of the thesis, we shift focus to a fully spiking model, without using the Extended Kalman Filter, and work with real sensor data from the University of Michigan's NCLT dataset. This dataset contains data from a Segway robot on campus, recorded using various sensors of differing quality. A comprehensive analysis is conducted to evaluate how each type of sensor impacts the accuracy of the model. Specifically, we assess the contribution of wheel encoders, IMUs, and gyroscopes to the model’s performance. The objective is to estimate the next position from the initial position using multiple sensors, correcting GPS errors. Commercial GPS systems are often prone to errors due to environmental interference, multipath effects, or satellite synchronization issues. To address these limitations, our fully SNN-based model integrates inputs from these various sensors, aiming to minimize GPS-induced errors and provide accurate position estimates. The proposed model determines the location of a moving object at each time step, relying on GPS when signals are strong and considering the previous prediction of the neuromorphic model when GPS is degraded. This allows continuous tracking in challenging environments. Results are compared with ground truth data from the NCLT dataset, obtained via a SLAM algorithm using LiDAR and high-quality GPS. Our spiking model shows up to a 13% improvement in accuracy over GPS alone, underscoring its effectiveness in difficult conditions. In conclusion, the work presented in this thesis demonstrates that SNN models can serve as a viable alternative to traditional deep learning models. It shows that SNN can achieve results comparable to state-of-the-art methods while offering significant advantages in terms of computational efficiency. This is because they can be deployed on neuromorphic HW, which has shown order-of-magnitude efficiency improvements when compared to standard architectures in terms of energy per inference.