Machine Learning-driven Management of Unmanned Aerial Vehicles Networks

In recent years, Unmanned Aerial Vehicles (UAVs) have been deeply investigated because of the relevant services they can support and the improvements they can provide in different application fields. Examples include last-mile delivery, area monitoring, and infrastructure inspection. UAV-aided communication networks can indeed extend or replace the existing communication infrastructure where such facilities would be difficult or too costly to deploy due to the remote or inaccessible locations, like in the case of areas hit by natural disasters. In spite of the recent advancements, UAV operations and scenarios introduce unique technical challenges, among which remote control and efficient usage of computational resources emerge as aspects of primary importance. In this context, data-driven approaches such as machine learning represent an effective methodology. UAVs, as well as ground stations with which UAVs communicate, can collect information on the tasks to be executed, the experienced channel propagation conditions, and the extension of the covered area – piece of information that can be leveraged for the design of effective solutions. Importantly, while developing algorithms to address the aforementioned issues, it is also critical to evaluate the system performance through scalable, yet realistic, simulations. In this work, both objectives are pursued, i.e., to develop and implement a machine learning solution that exploits the data collected by UAVs and ground stations in order to determine the best positioning and computing tasks assignment, while simulating the system in realistic settings. A deep reinforcement learning model is proposed and integrated with the well-known ns-3 simulator and with realistic channel propagation model for predicting the path loss between a low altitude platform and a terrestrial terminal. The prediction is based on the urban environment properties and is dependent on the elevation angle between the terminal and the platform, affecting the Line of Sight (LoS) condition of the communication link. To conclude, ns-3 offers a noteworthy simulation environment that allows to efficiently evaluate and compare several scenarios, protocols and applications. This permits to easily encompass all the different aspects that contribute to design quality and network performance. Future work could concentrate on improving the techniques tackled in this thesis and further extending the proposed scenario with other promising machine-learning-based frameworks that may improve the best action UAVs can perform to maximize an arbitrarily defined gain. The prospect of being able to merge realistic network simulations with innovative artificial intelligence optimization algorithms serves as a continuous incentive for future research that may also involve other cutting-edge scenarios.