Exploring the Potential of Generative AI in Media Production through Digital Humans

In recent years, Synthetic Humans and their applications have attracted considerable attention in a variety of fields, leading to an extensive exploration of their integration into Digital Twins, the Metaverse, and digital media production. This thesis explores the complexities involved in the digital human production using a semi-automated approach to find a fair trade-off between high-quality outputs and efficient production times, which is critical in a small and agile context. The study was conducted in collaboration with RAI, using their photo and video archives to retrieve images of relevant subjects for texturing and 3D reconstruction. The goal is to give a second life to RAI’s extensive archive of 2D footage and propose improvements to the media experience. After an overview of the state of the art of Synthetic Humans creation, this study proposes innovative strategies in order to (i) automate the identified workflow involving Unreal Engine 5 and MetaHuman Creator, and (ii) make it more versatile and modular. In this work, the improvements have been distributed among different stages of the digital human creation process, starting with the scripted generation of 3D head meshes from 2D input images of the reference subject using a Blender plugin and then moving on to the generation of suitable images for texture development using Stable Diffusion, conditioned on the fine-tuning of the trained models. These assets are in turn integrated into the Unreal Engine, where a developed widget facilitates the posing, rendering, and texturing of the MetaHumans. To complete the analysis, a thorough quantitative comparison between the subjects’ original images and the rendered MetaHumans material to ensure an objective assessment of their similarity. In addition, subjective tests were performed to validate the chosen objective metric. This work not only contributes to the field of Synthetic Humans and their application in the broadcast industry but also demonstrates the transformative potential of Generative AI in optimizing and enriching their creation workflow. The insights and methodologies presented in this work lay the groundwork for significant advancements in creating realistic, versatile, and fully personalized Virtual Presences.