AI Security Assessment: Attacks and Defenses on Large Language Models

The thesis activity aimed to provide an extensive overview of Large Language Models (LLMs), their usage in companies, and the associated vulnerabilities and security needs, emphasizing frameworks like MITRE, NIST, and OWASP's Top 10 LLM vulnerabilities. We started with an introduction to LLM architectures, including transformers, and discussed state-of-the-art techniques such as fine-tuning, reinforcement learning, retrieval-augmented generation (RAG), LLM agents, and prompt engineering. Then, we highlight how these technologies are widely used in companies utilizing LLMs. Key vulnerabilities are examined, with detailed examples such as prompt injection attacks, the widely used vector and unsafe output handling. To study vulnerabilities and frameworks, attacks on major public LLMs like GPT were conducted or existing ones were analyzed, providing insights into real-world implications and security measures. Defensive strategies and mitigation tools like Garak, LlamaGuard, and LLM Guard are evaluated and compared. Finally, examples of real use cases with proof-of-concept architectures and usage were conducted, highlighting attacks conducted on them and how the defensive tools can prevent those attacks. This work demonstrates how current AI-integrated architectures, as well as potential future implementations, can be highly susceptible to novel threats. These vulnerabilities can be exploited to execute various known or totally new cyber attacks. The focus is also on strategies to prevent and defend against these types of risks, which must be taken into consideration, creating a totally new work figure, fundamental in this field, the AI-Security Specialist, understanding the best practice or what to avoid in the process of deploying AI-based applications.