Predicting User Quality of Experience to Identify Network Issues in Live Streaming services: An ISP Perspective

In today’s world, video streaming accounts for a significant share of Internet traffic. Internet Service Providers (ISPs) strive to provide these services while ensuring the best possible Quality of Experience (QoE) for users. One of the biggest challenges for ISPs is that, unlike content providers who can directly access users’ QoE metrics, they can only derive them from network measurements such as throughput, latency and packet loss. This limitation is mainly due to end-to-end encryption, which limits the possibility of deep packet inspection, by enhancing user privacy while limiting insight into application-level metrics. The goal of this thesis is to develop methods that allow ISPs to accurately estimate user QoE relying on passive network traffic measurements that can be collected without breaking end-to-end encryption. This approach also makes it possible to identify problematic users with poor QoE while also allowing ISPs to determine whether service degradation originates from their infrastructure or is limited to specific users. To this end, I use machine learning techniques to predict the quality of video streaming and identify stall events based solely on passive network traffic measurements. The models are trained and evaluated on a dataset composed of traffic measurements describing video streaming content collected in a controlled environment and then tested on real-world traffic. In particular, this thesis analyses the streaming of DAZN videos, a popular sports streaming service. The real-world traffic was collected from an Italian Internet Service Provider. The personal data were properly anonymised to protect the privacy of the users. To identify QoE problems, the aim of this thesis is to predict changes in video resolution and stall events. For this purpose, starting from network logs that describe the data flows exchanged between a user and the content provider with passive measurements, I process millions of flows to extract the features required for the classifiers. Predictions are performed in 10-second time windows, which enables the identification of users with a high rate of resolution changes or rebuffering events, as well as users with consistently low video quality. Users with low QoE are further investigated by analysing additional network metrics such as RTT, the number of open flows and the amount of data exchanged to identify the main causes of QoE degradation. To visualise the results, a dashboard was developed that provides both aggregated and detailed views of QoE over time, while also allowing the examination of individual users and the associated network metrics. The results show that the proposed method can identify video resolution and stall events with high accuracy. After applying the developed method to the ISP data, I identified several problematic users where the ISP confirmed that they had connectivity issues.