Design and Development of a Dual-Task Exergame with Avatar-Based Coaching for Patients at Risk of Mild Cognitive Impairment

This master’s thesis is conducted within the framework of a broader European research initiative, the Dorian Gray project, exploring the correlation between Mild Cognitive Impairment (MCI) and Cardiovascular Diseases (CVDs). The European project adopts a multidomain approach that combines MCI risk monitoring with physical exercise and cognitive training. Recent studies have highlighted the role of CVD symptoms in damaging brain structures and its vascular network, thus suggesting that these may act as risk factors in the onset of dementia. Physical inactivity in the ageing population should be addressed as a key variable: it can double the risk of developing dementia later in life if not accounted for. In this context, the thesis project focuses on the gamification of physical and cognitive training to promote long-term engagement and adherence. Given the importance of targeting flexibility, strength, and cardiovascular fitness while simultaneously stimulating cognitive function, the proposed solution consists of an exergame implementing dual-task exercises. Specific fitness movements have been chosen to aim at full-body exercise, to be completed concurrently with memory and attention tasks. To ensure usability and optimal user experience, the exergame is designed for mobile devices, specifically for Android tablets (Samsung Galaxy Tab S9 FE or similar), and utilises the device’s video stream from the frontal camera. The QuickPose framework has been employed to integrate one of MediaPipe’s pose estimation models and gesture recognition techniques to track and classify physical movements. In addition, the project implements patient’s embodiment through a personalised avatar representing the user in the game environment according to their preferences. To further promote engagement, the tracked movements are employed to animate the avatar in real-time. Feedback mechanisms were designed to operate both in the short term, through timely responses during gameplay, and in the long term, by supporting the monitoring of progress over time. An avatar-based digital coach is predisposed to guide the patient through the lifestyle intervention, demonstrating the exercises to complete and encouraging correct execution. The coach provides supportive verbal feedback and remarks on movement form during task performance, further promoting engagement and long-term adherence. After development, a preliminary testing of the exergame was performed to evaluate its usability and investigate key features like immersion, presence, and user perceived competence. The system was tested with 27 predominantly young participants, largely with higher-education backgrounds. Usability assessment resulted in a median SUS score of 80 ± 11.37, placing the system in the Good–Excellent range and confirming its overall acceptability. Additionally, task performance showed significant improvement after the first trial (p < 0.001), indicating that participants understood the game directives and gained confidence with continued play. Furthermore, real-time avatar animation was highlighted as the most immersive feature, supporting engagement alongside the presence of the digital coach. This work integrates multiple design features to demonstrate the technical feasibility of engaging exergames, showing preliminary usability and pointing to immersive, presence-enhancing features as a promising topic for future research.