Pixel-level kernel estimation for blind super-resolution

Throughout the past several years, deep learning-based models have achieved success in super-resolution (SR). The majority of these works assume that low-resolution (LR) images are 'uniformly' degraded from their corresponding high-resolution (HR) images using predefined blur kernels - all regions of an image undergoing an identical degradation process. Furthermore, based on this assumption, there have been attempts to estimate the blur kernel of a given LR image, since correct kernel priors are known to be helpful in super-resolution. Although it has been known that blur kernels of real images are non-uniform (spatially varying), current kernel estimation algorithms are mostly done at image-level, estimating one kernel per image. These algorithms inevitably become sub-optimal in handling scenarios where an image is degraded non-uniformly. A divide-and-conquer form of approach, dividing an image into several patches for individual kernel estimation and SR can be a simple solution for this matter. Nevertheless, this approach fails in practice. In this paper, we address this issue by pixel-level kernel estimation. The three main components for training a SR framework based on pixel-level kernel estimation are as follows: Kernel Collage - a method for synthesizing non-uniformly degraded LR images, designed considering the coherency of kernels at neighboring regions while abruptly changing at times, the indirect loss - a novel loss for training the kernel estimator, based on the reconstruction loss, and an additional optimization - a scheme to robustify the SR network to minor errors in kernel estimations. Extensive experiments show the superiority of pixel-level kernel estimation in blind SR, surpassing state-of-the-art methods in terms of quantitative and qualitative results.