Towards meaningful Image Anonymization using Semiotic Analysis

In everyday life, we come in contact with an ever growing amount of data, making its every related aspect a matter of concern: where to retrieve data, where to store it, and how to organize it are some notable mentions, but also only a fraction of a much longer list. Among these challenges, respecting the privacy of the individuals captured in a media deserves the right attention. In our work, we focus on the Image Anonymization task, which aims to preserve the personal information of people depicted inside photographs by modifying them. There are many techniques to achieve this goal: the main ones include covering the people's faces, sometimes even entire bodies, with blur, solid colors, generated masks, or by recreating the entire picture from scratch, but these approaches come with the issues of maintaining the meaning of the original image, not compromising the overall quality when editing identifiable details, and achieving a good level of anonymization. Our architecture is an extension of the CAMOUFLaGE-Light model, where we make use of pretrained models to extract different types of information from a picture and generate an anonymized version based on a portion of the original and the obtained features: we employ FRESCO and RelTR to analyze the starting input and produce a data structure containing every information deemed relevant, IP-Adapter and T2I-Adapter to learn from the extracted features, and Stable Diffusion to reconstruct the de-personalized photograph. Particular attention is paid to distill information such as ethnicity, age, gaze direction, body pose, and similar characteristics that help to define the semiotic of the picture we want to obfuscate. We execute different types of tests to assess the quality of the final output, where image quality, re-identification rate, semiotic analysis, and downstream-task model performance are used as metrics to compare the output scores with other state-of-the-art methods. Finally, we discuss possible future developments that could bring further advances in the efficacy of the proposed method.