Multi-Role Simulation and Training for Forest Fire Management using Mixed Reality Wearable Devices

This thesis work takes its origin from an already existing project previously carried out by the VR@POLITO laboratory of the Department of Control and Computer Engineering at Politecnico di Torino in collaboration with Corpo Volontari Anticendi Boschivi (AIB) of the Piedmont region. In the previous iteration of the project, a tactical sand table used for the training in forest firefighting (located at the training center of the mentioned first responders¿ body) was interfaced with Cell2Fire, a physically-accurate forest wildfire simulator. On one side, it allowed to project onto the sand table, in spatial Augmented Reality (AR), a given case study to be analyzed and explained to the trainees. On the other side, the trainees were provided with a Mixed Reality (MR) application, capable of superimposing additional 3D contents to the sand table through hand-held (tablet) and head-mounted (Microsoft Hololens 2) MR devices. In particular, the trainees were able to move forward and backward in the simulation autonomously and to interact with it, by trying various combinations of firefighting interventions to test their effect on the simulation, in order to find the most effective and efficient method to address the fire related to the case study. The system, designed for being used in formal courses for various positions of command in forest firefighting, was also prepared for a future multi-user implementation, where more than one trainee at the same time might interact with the same simulation of the considered case study. Based on the outcomes mentioned above, this thesis work focused on redesigning the existing experience with a new, multi-role perspective. In fact, the instructor¿s figure has been included inside the application and his or her role will be to observe, in real-time, the trainees from his or her own device and interact with them, also by inserting the so called ¿injections¿ in the map. This may help to change the course of the planning phase of the training in the way deemed best for the trainees. The research purpose of this pedagogical protocol modification is to evaluate how the instructor, if rendered as an active element of the virtual experience, can positively impact the effectiveness of the training. At the same time, another important development point has been covered in this thesis: the exploration of the application potential by its deployment on video-through-based wearable devices. The Hololens 2 see-through experience turned out to be, on the one hand, very fluid since it lets the trainees view holograms by staying with their eyes perpetually anchored in the real world, but, from the other hand, its limited field of view doesn¿t allow them to have a complete and natural-like vision of the 3D contents, spread across the whole real sand table. Consequently, this new experimentation aimed to enhance the MR experience through the use of video see-through technology in place of the optical see-through one. Specifically, the Oculus Quest Pro standalone headset was employed, whereas a revamped version of both the User Interface (UI) and User eXperience (UX) were developed with the aim to improve the experience in terms of usability, ease of use, intuitiveness, and appeal (based on the feedback received by the final users). These efforts led to investigate which type of MR experience can provide the best results in terms of usability and training effectiveness, against the respective pros and cons of the two devices.