LUNA: Learning-based Unified Network Allocation AI-based NoC workload optimization

The design and optimization of a Network on-chip (NoC) for efficient communication in System-on-Chip (SoC) architectures pose a complex challenge. It involves balancing diverse requirements such as low latency, high bandwidth, and energy efficiency while managing the increasing complexity and heterogeneity of on-chip components. One of the most serious challenges is how to allocate the different computational workloads in the compute units of the NoC. The allocation process has a major impact on power consumption, performance, and of hardware resources required for correct, routing, and parallelization. Additionally, poor Network-on-Chip (NoC) design can lead to system deadlocks or livelocks, highlighting the need for more efficient and robust communication architectures in System-on-Chip (SoC) systems. While traditionally the allocation has been done using deterministic and rule-based algorithms, this research project proposes a novel alternative based on deep learning. The main objective of this study is to develop an artificial intelligence system dedicated to improving the efficiency of hardware accelerators. Reinforcement learning techniques are exploited for the intelligent allocation of neural network Computing Graphs on a hardware accelerator integrated with a NoC. In this exploratory work, the NoC is designed with uniform nodes capable of executing any function. Its configuration and the Directed Acyclic Graph (DAG) representation of Computing Graph, are characterized by a set of attributes associated with nodes and edges, and are constructed using the NetworkX library. The investigated frameworks for this purpose include TensorFlow and PyTorch, along with their specialized libraries, TF-Agents, and Cleanrl. The analysis will progress step by step, with a primary focus on how the neural network learns, reaching convergence in the parameters governing the allocation of the Computational Graphs on the NoC. The effectiveness of reinforcement learning enables to obtain positive outcomes in the learning phase. As an initial achievement, the node assignment was successfully determined based on a valid criterion. These findings form the basis for addressing more complex challenges in this innovative field. This approach guarantees clarity and transparency, offering valuable insights into how a deep reinforcement learning algorithm behaves and performs.