Optimization of Spiking Neural Networks execution on low-power microcontrollers

The rapid proliferation of AI has prompted researchers to direct significant attention towards the development of novel and innovative solutions that optimise its performance and power consumption. Spiking Neural Networks (SNN) represent a particular type of neural network that emulates the behaviour of the biological brain to enhance neural computation. This results in advantages such as low-power consumption, effective memory-processing colocation, and event-driven execution. The potential benefits of neuromorphic computing could be realised through the utilisation of optimised neuromorphic hardware, such as SpiNNaker 2 and Intel Loihi 2. However, these accelerators are difficult to obtain and often expensive due to their experimental nature. The present work investigates a potential solution to be implemented on microcontroller units (MCUs) to run SNN in small and low-power systems. To this end, SNN models are deployed leveraging HW modules such as DSP and memory access accelerators, available in the target architectures. The development focused on reproducing Leaky-Integrate and Fire (LIF) and Izhikevich neurons, implementing fully-connected, one-by-one, and recurrent connectivity. This effort led to the implementation of the snn2mcu C library which currently supports two neuron models, three connectivity patterns, and two target MCUs. The first target of the snn2mcu library is the general-purpouse ARM Cortex M7 core from STMicroelectronics. Quantization has been implemented converting floting point network parameters to fixed-point data types compatible with the ARM DSP library (CMSIS) that provides an optimised implementation of common math operations and vectorised functions. The second target is the open source GAP8 architecture with Risc-V processors. Optimization have been implemented by exploiting the multi-level memory layout and the multiple cores available in the system. snn2mcu also supports SNNs described with the Neuromorphic Intermediate Representation (NIR) framework. An automatic tool has been designed to generate optimized firmware starting from the high level NIR description of the SNN, thus removing the need for manual coding and facilitating development on embedded systems. To evaluate the usability of the library, two SNN models were deployed on the GAP 8 and ARM core. The first is a SNN designed to classify seven Braille characters using input spikes from a robotic finger that produces twelve signal channels. The second is a classifier for handwritten numbers taken from the MNIST dataset. The Braille SNN model on the ST-ARM was executed 6x faster than floating point SNN simulators achieving the same accuracy of 91.43% over a 140-sample test dataset. The second model achieved an average energy consumed per sample of 10.54 mJ and an execution time of 187.82 ms per sample, and power consumption in line with a specific optimised FPGA design. In summary, the present work demonstrates that the utilisation of optimisation techniques, such as fixed-point mathematics with reduced bit-depth, DSP vectorised functions, multilevel cache, and code efficiency, can enable an effective deployment of SNNs on a wide variety of commercial grade MCUs.