Online model selection for LLMs with limited annotations

During the last years, the exponential growth of machine learning and deep learning applications has remarked the need for efficient and scalable model selection strategies. Traditional model selection approaches often rely on extensive offline evaluation by means of predefined performance metrics. However, modern real-world applications typically involve non-stationary or streaming data, making it extremely complex to select the best model a priori. Online model selection has emerged as a crucial paradigm that enables real-time evaluation and adaptation of model selection based on incoming data streams. Industrial examples include autonomous systems such as robotics or self-driving cars, whose tasks can involve sensor fusion, real-time path planning and obstacle avoidance. From online advertising and recommendation systems to climate science and environmental pollution, from cybersecurity and intrusion detection to healthcare and medical diagnosis, most of these fields now employ pre-trained models or AI agents to improve their quality. With the advent of Large Language Models (LLMs), an increased demand for and reliance on labeled data is required. Unlike unlabeled data, which can be easily accessed from a variety of sources and can be collected almost inexpensively, labeled data require human effort, time and financial resources to be obtained. Furthermore, in an online setting, where a significant portion of data may not be available from the outset, continuous and non-scalable human effort would be required, which clearly represents a bottleneck in the selection process of large models. The core of this research is the evaluation of various adaptive and non-adaptive online model selection algorithms in the field of LLMs. While some previous work has covered the classification framework, this work brings an exclusive focus on generative tasks such as text summarization, machine translation and in-context question answering. First, we want to understand whether the state-of-the-art methods for classification setting can also perform well in a generative framework. The goal is to identify the best LLM from a pool of pre-trained models with the least number of annotations, separately for each data stream. Specifically, the starting point of this thesis is based on the strength of the Model Picker, an adaptive algorithm for pre-trained classifiers. This work covers the entire workflow, from the adaptation of the method to properly perform in generative modelling, to a deep and exhaustive exploration of further improvements. The outcome of this research outlines that the proposed method is robust and well performing in the generative framework, leading to possible openings in the field of online active model selection for generative language models.