Augmented Reality for the visualization of flight information coming from a model aircraft

The PhotoNext project consists in retrieving and processing information coming from a set of Fiber Bragg Grating (FBG) sensors, so that its status can be visualized in a simple and immediate way through a heat-mapped 3D model. It is carried out in collaboration with the Department of Control and Computer Engineering (DAUIN) and the Department of Mechanical and Aerospace Engineering (DIMEAS) of the Politecnico di Torino, as a part of the Inter-Departmental Centerfor Photonic technologies. The entire architecture is primarily developed to analyze and monitor a model aircraft created by the Icarus team in Politecnico di Torino. FBG sensors are devices that exploits photosensitivity: they measure the variation of the refractive index of the optical fiber core caused by some physical effect such as strain, pressure and temperature. This technology is particularly used as part of a preventive monitoring system. The flight information is displayed through an application designed for Augmented Reality using Microsoft HoloLens 2 headset called HoloLens Viewer. This allows the user to have an overview of both the aircraft during flight but also of the digital information coming from the sensors for real-time monitoring, or offline with an old collection retrieved from the database to analyze a past session. The aim of this thesis is to design an improved visualization framework exclusively for the HoloLens 2 device, in order to unify the two existing versions in a single enhanced version that can be used in real-time during a flight session. During the monitoring session the program shows information in two different ways. The first give an immediate visual feedback in the 3D model: for each placed sensor there are heat-maps that show its data in real time, with different colors based on its intensity and type. The second is a more technical view: a graph is shown with the wavelength values for each sensor over time, updated in real-time to allow the pilot or co-pilot to analyze the flight data trend. Since the app should work regardless of the type of model in which the sensors are placed, it is possible to import different 3D models, and to increase the ease of use it is possible to save a configuration where the sensors are placed in the model to reuse it later. A configuration file with some information is stored in the memory: HoloLens Viewer will read the contents of this file to automatically set some parameters, so that the user can avoid manually setting his system data while wearing the headset and immediately activate the display. The network connection of the application can be done via database or TCP protocol. In the latter case, it communicates directly with the middleware in charge of retrieving the data from the sensors. The final tests of the HoloLens Viewer monitored the usage of the CPU, GPU, memory and the number of frames per second. Several scenarios were analyzed. Using a wing and the tail of a model aircraft, created by the Icarus team, it was possible to analyze the real behavior, although tests were performed in the laboratory and not during a flight. Other analyzes were performed using a software that emulates the behavior of FBG sensors, which was created with the aim of simplifying test procedure. HoloLens Viewer showed nice results in term of performance. It was also tested by some members of the Icarus team, receiving satisfactory feedback.