OSCAR: Optimizing Architecture Search with Constraints-Aware Deep Reinforcement Learning

Despite its transformative potential, the adoption of Deep Learning remains very limited in resource-limited applications, mainly because of the need for significant expertise, personal experience, and intuitive understanding for an effective design of architectures. With the theoretical understanding of neural architectures still in its nascent stages, designing deep models often entails an empirical approach of testing a multitude of architectures – a process that is simply impractical in many scenarios. The research community has made concerted efforts in recent years to streamline and automate this critical yet often tedious aspect of AI research. The field of Neural Architecture Search (NAS) specifically focuses on this issue. In this thesis, we present a contribution in this direction, blending the latest advancements in Training-Free NAS with an intrinsically hardware-aware algorithm based on Deep Reinforcement Learning (DRL). First, we explore an integration of DRL within Genetic Algorithms (GA), demonstrating the ability of ad-hoc controllers to optimally modify candidate architectures using test-accuracy proxies rather than relying on costly evaluations of post-training model performance. Considering the limitations in transferring solutions obtained from this hybrid approach across the various possible devices we considered in this work, we also propose a purely RL-based search algorithm optimized for operational hardware performance. This algorithm is built on real-world measurements of the latency on six different target devices—ranging from a Raspberry 4 up to an Eyeriss accelerator—instead of hardware-agnostic indicators such as the number of parameters in each model. Our methodology shows promising results across various performance indicators. In particular, this work presents a purely RL-based algorithm capable of designing tailored networks for distinct devices, effectively integrating operational hardware information into its decision-making process. This work aims at providing a pioneering stride in efficient-AI, and, to the best of our knowledge, showcases the first truly hardware-aware training-free RL-based approach to NAS. This research offers a more accessible and cost-effective approach to DNN design, contributing to the democratization of AI technology and to an extended applicability of Deep Learning in resource-constrained environments, such as aerospace applications and autonomous robotics.