Adaptive Validation of BPMN Exercises: Integrating LLMs for Generating and Assessing Equivalent Solutions

This thesis tackles the problem of offering dependable and scalable support for automated assessment in education by examining the use of Large Language Models (LLMs) in the validation of BPMN (Business Process Model and Notation) exercises. Designing and testing a methodological framework that can produce BPMN models that are structurally compliant and appropriate for methodical evaluation across several quality characteristics is the goal. To provide BPMN-compliant ground realities, a dataset of 24 heterogeneous exercises was created. With the help of an extended taxonomy and specific modelling principles, the approach developed into a two-step pipeline through iterative improvements: organised identification of BPMN elements followed by XML rendering. Overall strong performance was highlighted by evaluation across eight quality parameters, with six exercises attaining the maximum score of 100 and an average score of 93.15. While dimensions like ranking (mean 99.0) and traceability (mean 99.7) showed almost perfect stability, correctness (mean 92.8), completeness (mean 92.8), and consistency (mean 90.4) showed more variability, often falling below 70 for more complicated exercises. While certain dimensions, such as ranking and verifiability, operated more independently, correlation analysis demonstrated that these three dimensions are highly interrelated. These results show that BPMN validation frameworks can successfully incorporate LLMs. The suggested strategy highlights the advantages and structural drawbacks of existing models in managing challenging process modelling tasks, while providing a reproducible means of assisting with instruction and evaluation.