Patch-based learning of space-variant hyperparameters in variational image restoration.

Solving an inverse imaging problem is the task of reconstructing an unknown image from observed, often incomplete noisy data. A standard example describing a linear degradation process is the problem of image deconvolution where the operator describing the blurring process (forward operator) is a convolution matrix. These problems are typically ill-posed, where solving the inverse formulation can cause small perturbations in the observed image (input) to lead to significant perturbations in the desired image (output). A standard paradigm overcoming such instabilities consists in considering a regularised formulation of the problem in order to guarantee stable reconstruction of the solution. Regularisation is combined with a data-fitting term along with a scalar or vector of hyperparameters balancing the two. Total Variation regularisation is a popular regulariser in the context of imaging due to its enhanced edge-preserving behaviour and noise smoothing properties in comparison to other, simpler, regularisations (e.g. Tikhonov, l_1 norm). From a Bayesian perspective, Total Variation (TV) regularisation implicitly assumes a space-invariant, one-parameter half-Laplacian distribution (hLd) for the gradient magnitudes of the true image. However, this assumption is often restrictive in the modelling of natural images, whose statistics are often characterised by heterogeneous and highly oscillating behaviour. Ideally, in smooth regions, stronger regularisation is desired, while in textured areas, less regularisation is preferred. This led us to a new formulation: where, for each pixel, a different regularisation parameter has to be estimated to adapt to local image contents. This model better captures the space-variant (SV) nature of gradient distribution throughout the image. The derived approach can be efficiently solved by means of accelerated first-order algorithms such as, e.g., the Fast Iterative Thresholding Algorithm (FISTA) endowed with a non-monotone backtracking strategy to select optimal step-sizes. Differently from the scalar case, where in most cases a brute-force approach is used to select a optimal parameter maximising suitable quality metrics (e.g., PSNR, SSIM), in the SV approach such strategy is not feasible due to the potentially large size of the image considered. To mitigate this problem, we consider in this work a patch-based approach where, for each image patch (of fixed size) a golden section algorithm is used to select an optimal scalar parameter by suitable comparison with ground truth data. The value computed can be then used as an optimal value in the centre of the selected patch and the map of space-variant parameters can be built in a sliding window fashion. By using natural image datasets of different size and noise distribution/intensity, we then implemented a supervised learning approach where a light neural network was trained to predict the optimal parameter in a pixel looking at a small patch around the pixel itself, leading to fast and effective inference after suitable training . The space-variant reconstruction approach obtained better results in terms of SSIM and PSNR with respect to the global reconstruction leading to better detail preservation and reduced over-smoothing in the textured region. Also, our ML approach drastically decreased the computational time required to compute the parameters map without the use of ground truth, an essential feature in real-world applications.