A Conversational Agent to Support Virtual Reality-based Training on the Life-saving Intraosseous Insertion Procedure

Nowadays, training on the Intraosseous Insertion (IO), a critical life-saving procedure in the medical field, is a significant challenge. Mastering this technique, which requires high psychomotor precision and knowledge, is necessary to preserve the patient’s life. The aim of this thesis work was the development of a virtual assistant based on the open-source Rasa framework, to improve the training of healthcare professionals and paramedics in the IO procedure. The main objective was to provide an interactive and effective learning experience through the use of a conversational agent specifically designed to understand users’ requests and provide contextualized instructions, suggestions and feedback during the execution of the procedure. Once developed, the conversational agent was then incorporated into a serious game in Virtual Reality (VR), adding voice commands, to help the users with any issues or questions they may have as they move through the experience using natural language interaction. The architecture of the Rasa framework, the dataset used for training the underlying AI, as well as the connection with the Unity platform supporting the VR experience and, therefore, the integration of the conversational agent in the serious game will be discussed. The evaluation aimed at assessing the user experience during the serious game session in terms of usability will be presented too, as it enabled the collection of information that may be used in the future to add new functionalities and explore new usage scenarios.