Performance evaluation and design of ML-based solutions for the support of mobile services in 5G systems

The vision of the fifth generation of mobile networks (5G) lies in bringing enhanced performances with respect to the previous mobile technologies. 5G has been planned to provide revolutionary high throughput, extremely low latency, for miscellaneous devices with massive and ubiquitous connections. To achieve these goals, many advanced technologies have been introduced and utilized in 5G, like massive MIMO, mmWave, efficient Radio Resource Management (RRM) techniques, etc. Among all, an efficient RRM could have a significant impact on effective spectrum utilization, massive connections. One thriving solution is represented by virtual Radio Access Network (vRAN) technology and is profitable in terms of cost and scalability for the mobile network operators. Indeed, in virtualizing the RAN, radio processing intelligence, which was initially performed by purpose-built hardware, will be performed at higher levels of the network. vRAN adds the ability to scale the network resources assigned to the various demanding entities in 5G network. This is achieved by separating networking functions from hardware, a technique that overcomes many technical challenges in the integration with legacy technologies. Although vRAN has brought flexibility and cost reduction in terms of hardware, it has introduced new challenges. Indeed, by increasing the number of connected devices with heterogeneous demands to vRANs, the network performance will deteriorate. Despite the mentioned issues, the conventional mechanisms might not be sufficient to fulfill the optimal performance and resource allocation. Therefore, a more efficient and intelligent RRM is required, which can dynamically scale and allocate radio resources. Therefore, in recent years Machine Learning (ML) and in particular Reinforcement Learning (RL), has introduced versatile applications in the telecommunication area, and several intelligent resource allocation mechanisms have been proposed. In this regard, the main objective of this thesis is to design and to study online learning mechanisms based on RL-based and deep RL-based RRM for vRAN, to analyze their performance in typical network scenarios, and to compare the possible enhancement of both methods. The proposed solutions are aimed at real-time and dynamic RRM according to user demands in vRANs and they are designed to deal with the dynamics associated with the radio environment. The simulations have been carried out using the Network Simulator 3 to prepare network scenarios for the generation of traffic between vRAN and multiple users receiving different channel conditions. For the main scenario, it is considered to implement not only multi-user but also non-stationary users, in presence of one eNodeB. This scenario allows analyzing the proposed approaches under more complex situations. The first approach is grounded on a differential semi-gradient State-Action-Reward-State-Action (SARSA) mechanism, and the second approach is based on Deep Q Learning (DQL) mechanism for real-time RRM in vRAN. The RL agent is responsible to receive the channel conditions, which are considered as Channel Quality Indicator (CQI) and transmission buffer head of line delay. Then following a greedy policy, the agent selects optimal Modulation and Coding Scheme (MCS) values to maximize the average reward which denotes the user's throughput. The proposed RL-based solutions are promising and can meet the demands of the heterogeneous spectrum of users.