Natural Querying for Travel Industry: Bridging LLMs and Data

The travel industry relies heavily on large volumes of data, requiring efficient methods to extract insights from big and heterogeneous datasets. This thesis presents a novel approach to link Large Language Models (LLMs) with SQL databases to enable natural language querying, facilitating more intuitive and accessible data interactions for all user profiles. The research is conducted in collaboration with Amadeus IT Group and focuses on developing a co-pilot tool that interprets user queries, formulates corresponding SQL queries, and returns their results while assisting the user. The core challenge addressed in this work is the inherent ambiguity present in natural language queries. We investigate ambiguity on different levels proposing techniques to detect and handle them. By leveraging an LLM-powered agent with tool calling functionality and prompt engineering strategies, the system generates multiple SQL interpretations for ambiguous queries trying to capture as much as possible the intention of the user. To evaluate our agent, test cases for ambiguous, unambiguous, and unanswerable questions have been employed, a well-known dataset and a generated test suite over Amadeus Passenger Name Record (PNR) database. Evaluation using state-of-the-art LLMs demonstrates improved performance in ambiguity detection and resolution when compared to baseline models. Additionally, testing on both unambiguous and ambiguous cases highlights the extreme difficulty of this task for models, particularly when applied to industrial databases. On average, performance decreases by approximately 60% when transitioning from unambiguous to ambiguous settings. Our analysis contributes to the growing field of text-to-SQL systems by highlighting the importance of ambiguity management and the effectiveness of agent-based LLM solutions in real-world applications.