InsureAI: Leveraging LLM-Powered Intelligence for Efficient Insurance Complaint Processing

Large Language Models (LLMs) have revolutionized natural language processing tasks, showcasing impressive abilities across various domains. This thesis explores their application within the insurance sector, focusing on tasks like classification and information extraction. The research centers on developing a system named InsureAI, which utilizes LLMs to automatically categorize insurance complaints across multiple levels and extract pertinent information from these complaints. The goal is to investigate the effectiveness of two different models - Llama-70b and Zephyr-7b - by employing two different approaches for implementing this system: zero-shot learning and instruction fine-tuning. The former uses the pre-trained knowledge of the model to perform inference while the latter further trains the LLM explicitly on a custom dataset of insurance complaints tailoring the model to the domain-specific dataset. The process begins with obtaining raw data of insurance complaints (the source). This data undergoes thorough exploratory data analysis (EDA), with a focus on understanding the distribution of classes for hierarchical classification tasks. Pre-processing steps follow, which entail identifying authentic insurance complaints, filtering out any extraneous noise, and standardizing the ground-truth information to construct a clean dataset. Extensive experimentation is conducted to evaluate and compare the models. This involves comparing the predictions generated by the models with the ground-truth data, using the Semantic Answer Similarity Score metric. Results indicate that the fine-tuning approach outperforms the zero-shot approach in both tasks, demonstrating superior comprehension of insurance complaints and using minimalist prompts to minimize out-of-memory errors. In conclusion, this thesis offers insights into leveraging zero-shot and fine-tuning techniques with LLMs. The findings highlight the potential of LLM-based approaches to enhance decision-making processes and optimize workflow efficiencies in the insurance sector.