Augmented reality to support motion capture shooting

Technological improvements have radically changed the way films involving VFX as well as video games are made. From green screen to ledwalls, from keyframe animation to motion capture, from paper storyboards to 3D visual storyboards, from normal cameras to simulcams and so on. Today, it is much easier to use technology to reduce costs and improve the quality of these products. On the other hand, all these changes have confronted actors with new scenarios, such as acting in the presence of green backgrounds or other actors wearing motion capture suits, which could be real challenges for them and add a greater workload in terms of imagination to understand what their character is actually seeing and feeling. This thesis aims to demonstrate how augmented reality can be useful for actors when shooting scenes in which motion capture is involved. The idea is to have the actor use an OST-HMD (Optical See-Through Head-Mounted Display) that allows him to see, in real time, a 3D virtual avatar superimposed on the actor who is controlling him through the motion capture suit, together with the other real and virtual elements of the scene. In this way, the actor could become more familiar with the virtual character with whom he or she is interacting and with the environment in general, better understand what the character being played is facing and feeling, and thus, hopefully, play the scene better. To this end, a framework was created that, using a visor of the indicated type (in particular, a Microsoft HoloLens device) and the Optitrack motion capture system, allows the user wearing the visor to see in real time an animated avatar superimposed on the actor traced by the optical system. In order to show the usefulness of the proposed solution, an experiment was conducted involving a number of users. The experiment consisted in proposing the script of a scene to some volunteers, who had to prepare to act out the scene through rehearsals conducted either with a traditional approach (i.e., using only props) or with the proposed tool (i.e., using the augmented reality viewer). The acquired subjective and objective data were analysed and compared. The results obtained will allow further steps to be taken in this promising field of research.