AI Driven Automatic Feedback Loop in Virtual Reality Serious Games Based on Target Stress Level

The aim of this thesis work was to design and develop a fully adaptive feedback loop system for virtual reality that uses real-time physiological signals to automatically modulate game experiences according to predicted user stress levels. The system comprises a context-independent plugin that allows a game application to automatically adapt in-game elements, balancing the user's state and maintaining a designer-specified "target stress level". It uses a local server for data processing and real-time stress assessment, taking GSR biosignal and producing a binary prediction value ("non-stressed" or "stressed"); the inference result is then streamed to all connected VR applications. As part of the work, the deep neural network for automatic stress recognition was designed, trained, and tested. When the feedback loop subsystem receives the prediction value, it uses it to update a custom representation of the user's current stress value and decides what feedback to produce (or not produce) inside the VR app. In the work, characteristics of different signals and their relationship with the "mental model" used to represent the user's current stress level were investigated. A prototype of feedback loop integration was then designed and implemented within an ad hoc test application in VR. The test scenario is based on successive time-limited tasks targeted to provoke acute (performance-related) stress, derived from exposure to challenging situations. The application was tested by validating the feedback elements' ability to reduce or increase stress systematically, comparing the scenario objective with model outputs and users' perceived stress levels. This work has potential applications in physiotherapy, rehabilitation, and training in VR, where user-centered adaptive experiences are valuable. For example, known limitations in VR rehabilitation pathways would be handled by adaptive experience elements: the application would automatically adapt itself to a user's stress level, providing healthier sessions and avoiding patient frustration or feeling unwell. The work was done in collaboration with University College of Cork (UCC), as part of a thesis abroad programme.