Nuclear Security: A Natural Language Processing Generative Approach

This thesis aims at investigating and evaluating the use of natural language gen- erative models, specifically in the framework of generating training scenarios for personnel working in critical infrastructures. Safety and security of an infrastruc- ture containing radioactive material should be addressed with emergency planning, which requires training scenarios for personnel involved. Such scenarios are tra- ditionally developed by human experts, although this is a process that is subject to several drawbacks. First of all, it requires a considerable amount of time for producing a qualitative output. Second, it must deal with multiple repetitive steps, thus leading to critical bottlenecks. Third, a final training scenario can at first glance seem reproducible and on-point, but when it comes to put it in practice, its plausibility might be insufficient. These aspects can ultimately lead to considerable decrease in training quality, with severe consequences on overall safety and security. With this in mind, the possibility of using generative methods in such pipeline has been explored in this work. Generative methods employ the use of Large Language Models (LLMs) to generate long and meaningful text from a simple user prompt. These models leverage the power of Machine Learning (ML) techniques to infer commonly occurring patterns in large training datasets. The use of generative models in scenario development has the potential of stream- lining the most tedious steps, thus improving quality and reliability for the final result. Their use can be investigated with two methodologies in mind, where a fully automated or a semi automated framework are both available solutions. In order to assess the quality of the scenario, some key evaluation criteria were selected and defined. After experimenting with multiple implementations and exploring existing literature, a hierarchical framework using a Generative Pre-trained Transformer (GPT) model was developed, with the aim of generating meaningful, complete and usable scenarios. Multiple scenarios were extracted from several open source examples found on internet archives, in particular from drills designed by interna- tional agencies. Human experts were then asked to provide a score for each selected evaluation criterion. What has been observed after the scenario generation is that using a simple prompt, with the bare model, lead to lack of important information, such as a detailed timeline of the exercise. After providing a general structure, scenarios automatically generated presented more adherence to scenarios designed by humans. When Chain of Thought was used as a prompting strategy, outputs presented more detail and adherence to the selected structure. As a conclusion, the GPT model was able to generate meaningful scenarios, allowing for flexibility in its implementation. When adopting the hierarchical architecture, explicitly describing the context and limiting the working window helped the model to perform better according to the evaluation criteria.