Designing an end-to-end Pipeline for Developing and Deploying IoT Solutions on Embedded Neuromorphic Platforms

The primary objective of this thesis is to explore, develop, and evaluate solutions for building and training SNN-based models that are immediately compatible for deployment on state-of-the-art neuromorphic embedded systems. To extend the study to real use cases, this work specifically addresses the task of Human Activity Recognition (HAR). HAR involves the detection of human actions by analyzing motion data collected from sensors within the Inertial Measurement Unit (IMU) of smartphones and smartwatches. In particular, the thesis focuses on the revised version of the Wireless Sensor Data Mining (WISDM) dataset, comprising recordings from the accelerometer and gyroscope sensors embedded in IMUs of smartphones and smartwatches. Historically, the feasibility of HAR has been affected by the limited computational and power resources available in portable devices, making real-time data processing impractical. The ultra-low-power and ultra-low-latency capabilities of Spiking Neural Networks (SNNs), which can operate on non-von Neumann neuromorphic devices, offer a potential solution to these challenges. SNNs model their behavior on biologically inspired computation, mimicking the mechanisms of organic brains. This approach is of high interest due to its impressive computational capabilities, low latency, and minimal energy consumption. SNNs consist of spiking neurons that operate in a sparse mode, in contrast to the continuous activity of neurons in traditional Artificial Neural Networks. Each neuron processes sparse input events, known as spikes, and remains inactive in the absence of input, making them unparalleled in terms of energy efficiency. This thesis specifically targets two neuromorphic embedded systems: DYNAP-SE2, developed by the Institute for Neuroinformatics of Zürich, and Xylo, developed by SynSense. DYNAP-SE2 is a mixed-signal asynchronous chip that utilizes analog spiking neurons and synapses for computation. Xylo is a fully digital synchronous chip tailored for processing low-dimensional input signals. The overall design, development, and training of solutions for these devices pose considerable challenges, mainly due to the unique characteristics and limitations of each platform. This thesis proposes an end-to-end multi-step pipeline for generating models compatible with the target devices, that can be generalized to work across a wide range of applications. The ultimate goal of the thesis is to gather information about the capabilities of target devices by evaluating their performance under varying levels of input data complexity. Both devices are benchmarked across a series of eight tasks of increasing complexity, based on different subsets of the full HAR task, to discover their behavior near their operational limits. This work introduces critical techniques, such as Synapse Pruning and Quantization Tuning, which enhance the utilization of onboard chip resources and overall model performance. In particular, Quantization Tuning has proven to be necessary to ensure consistent performances after hardware deployment. The proposed pipeline can produce well-trained, hardware-ready models that can be directly deployed on the target devices. DYNAP-SE2 and Xylo have proven to be mature enough for simple and medium-difficulty tasks, but struggled with more complex ones primarily due to limited software support. Improving the software support will be critical for unlocking their full potential and extending their applicability in real-world scenarios.