Artifacts Segmentation with Convolutional Neural Networks for smartphone camera Image Signal Processor

In recent years, deep learning has garnered significant interest due to its ability to achieve state-of-the-art results across a wide range of applications, including image processing. By leveraging powerful algorithms and vast amounts of data, deep learning has produced remarkable results in tasks such as image classification, object detection and semantic segmentation, through Convolutional Neural Networks architectures. This is due to their capabilities in learning high-level representations of the data, capturing the underlying patterns and extracting features from the images. As a result, traditional image processing algorithms have been outperformed by Artificial Intelligence techniques in terms of accuracy and efficiency, and researchers have been able to tackle more challenging and complex tasks with greater ease. At Huawei Nice Research Center the teams are working intensely on the smartphone camera image signal processor, developing efficient solutions which can be integrated into the next generations of smartphones. This thesis work aims to demonstrate how Convolutional Neural Networks can be used to detect image artifacts at the pixel level that may be created in the image signal processor. First, by focusing on the demosaicing and ghosting artifacts, which are known to have a significant impact on image quality, the study builds the datasets labels for the training of deep learning models capable of detecting these artifacts, in a weak-supervised scenario. Then, the models performances are rigorously assessed on a variety of images, ensuring their robustness and testing their generalization capabilities. The ultimate goal is to use the trained models to create a useful metric based on artifacts detection, which can be used to assess the quality of images and to aid in the development of more effective image processing tools. Achieved results are promising since the trained models are using a limited number of parameters and they are able to correctly detect common image artifacts with a low inference time. The models have also been successfully used to optimize the parameters of the well-known Malvar-He-Cutler demosaicing algorithm which is an important improvement that may be applied to smartphone camera image signal processor.