Enhancing Cybersecurity with LLMs: Automated Identification and Risk Scoring of Network Misconfigurations

In a corporate environment where industrial secrets and critical assets need protection, ensuring robust security is essential. Misconfigurations, improper setups in systems or applications, can create exploitable vulnerabilities. These issues often derive from overlooked security best practices, such as unchanged default settings, excessive permissions, or wrong protocol configurations. This master’s thesis explores an AI-driven solution that leverages Natural Language Processing (NLP) techniques and Large Language Models (LLMs) to mitigate security risks. The aim is to develop a tool that not only assists users in analyzing network configurations and resolving issues, but also focuses on anticipating and preventing attacks by proactively managing security risks. The process begins with user-uploaded security documentation, from which best practices are extracted using a Retrieval-Augmented Generation (RAG) mechanism powered by a FAISS vector database. Next, an LLM generates a structured JSON template based on these best practices, defining a standardized format. The system then fills this template with network details provided by the user, and supplies them to a Llama-based chat-bot, which can perform a detailed security analysis and guide the user in troubleshooting misconfigurations. To enhance reliability and prevent AI hallucinations, the system incorporates few-shot prompt engineering and controlled temperature settings, ensuring that responses remain accurate. The key innovation is the introduction of a multi-LLM system designed to bridge the gap between flexible misconfigurations and rigorously cataloged security weaknesses and vulnerabilities. These models compute cosine similarity between extracted best practices and known weaknesses cataloged in CWE (Common Weakness Enumeration). This approach allows misconfigurations to be mapped directly to specific weaknesses, which can then be linked to known CVEs (Common Vulnerabilities and Exposures). The effectiveness is enhanced by fine-tuning two of the three used models on a specialized cybersecurity dataset, which contains paraphrased security best practices, in order to help the models recognize the semantic nuances specific to cybersecurity.