Protecting Privacy online using Latent Diffusion Models

Image anonymization is a challenging task that aims at removing all personally identifiable information to ensure the anonymity of the user who published the image while preserving the same semantic contents. The method proposed in this work is based on Latent Diffusion Models and exploits auxiliary networks to enable conditioning from different image-based modalities, which provides additional spatial information, without requiring to re-train the diffusion model. The approach involves two phases: real image decomposition and synthetic image reconstruction. The decomposition phase aims at extracting some non-sensitive representations from the original image using a set of task-specific pre-trained models and image processing algorithms. These information are then exploited in the reconstruction phase to force the generative model to preserve the content and the composition of the image. The proposed strategy allows to overcome a significant limitation of current state-of- the-art generative anonymization methods based on inpainting. The scene plays an important role in the image re-identification making it possible to retrieve the original image even if people inside are effectively obfuscated. Instead, by recreating the whole image from scratch, exploiting only some non-sensitive data from the original version, slight mutations are also introduced in the scene improving anonymization guarantees. Moreover, the use of multiple controls from different domains enables an an ¿ensemble effect¿ which allows to obtain a stronger control that compensates any weaknesses of each individual control, helping against artifacts and hence making the generation process unlikely to fail. As result, an high-fidelity synthetic version of an image collection that retains the same distribution and characteristics of the original images, in terms of visual content and semantics, can be generated.