polito.it
Politecnico di Torino (logo)

Unlocking the potential of Large Language Models in insurance domain: a case study

Mariangela Avantaggiato

Unlocking the potential of Large Language Models in insurance domain: a case study.

Rel. Lia Morra, Fabrizio Lamberti. Politecnico di Torino, Corso di laurea magistrale in Ingegneria Gestionale (Engineering And Management), 2024

Abstract:

Large language models (LLMs) have emerged as powerful tools in natural language processing showing outstanding performance across a spectrum of tasks and garnering considerable attention in research. Their capacity to generate human-like language and potential to reshape science and technology have encouraged their exploration across various industries, including insurance. The insurance sector faces the challenge of managing vast volumes of data from both internal and external sources and therefore formulating effective strategies for data management becomes crucial for maintaining competitiveness in an evolving marketplace. This thesis investigates the application of LLMs in classification and information extraction tasks within the context of complaints management. Specifically, the study aims to develop a system capable of autonomously extracting and interpreting information from customer complaints received via email. Leveraging two pre-trained open-source language model, Zephyr-7b-beta and Llama-70b, the research employs two training approaches to align the model with domain-specific knowledge. The first approach utilizes in-context zero-shot learning, enabling the model to perform targeted behaviors, without extensive retraining, trough configuring prompt statements. The second approach relies on Parameter Efficient Fine-Tuning (PEFT) for which the training phase involves adjusting the parameters of the last layer through a supervised approach. Starting from raw data and after extensive data analysis and pre-processing, a series of experiments were conducted on a test set comprising 1321 data points, selected through a stratified split. The models and two methods were evaluated using the Semantic Answer Similarity score metric. The findings demonstrate the positive impact of implementing LLMs in the complaints process, with fine-tuned model outperforming zero-shot learning approach for both classification and information extraction tasks leading to model accuracy of 80%. Through these methodologies, this work seeks to enhance the efficiency and accuracy of complaints management processes in the insurance industry as it places the ground for a further exploration that can be used to assess the eventual benefits of implementing the large language models to other case studies.

Relatori: Lia Morra, Fabrizio Lamberti
Anno accademico: 2023/24
Tipo di pubblicazione: Elettronica
Numero di pagine: 146
Informazioni aggiuntive: Tesi secretata. Fulltext non presente
Soggetti:
Corso di laurea: Corso di laurea magistrale in Ingegneria Gestionale (Engineering And Management)
Classe di laurea: Nuovo ordinamento > Laurea magistrale > LM-31 - INGEGNERIA GESTIONALE
Aziende collaboratrici: REALE MUTUA ASSICURAZIONI
URI: http://webthesis.biblio.polito.it/id/eprint/30625
Modifica (riservato agli operatori) Modifica (riservato agli operatori)