Adaptive Validation of UCD Exercises: Integrating LLMs for Generating and Assessing Equivalent Solutions.

The increasing use of large language models (LLMs) in software engineering has opened up new avenues for automating complex modeling tasks, particularly in transforming natural language specifications into formal representations. This thesis studies the adaptive control of UML Use Case Diagram (UCD) exercises using LLMs, such as ChatGpt and DeepSeek, to generate analogous solutions and evaluate student tasks. A set of 23 manually selected exercises, each with human-written reference solutions in visual and JSON formats, serves as the basis for the empirical investigation. Using prompt engineering methodologies (role-based and iterative prompts), LLMs were prompted to generate UCDs, which were evaluated using a multifaceted framework in relation to syntactic correctness, semantic equivalence and pragmatic usefulness. The evaluation employs systematic measures to compare the results of LLM models with validated references and analyze their completeness and redundancy. The results show that, although LLMs possess adequate syntactic capability, their semantic flexibility is limited, especially in learning environments where variable but correct solutions are expected. This study contributes a reproducible workflow and a new testing framework that highlights the advantages and disadvantages of LLMs in supporting modeling education, offering pathways for integrating adaptive feedback systems and enhancing automated assessment tools.