Find The Way Back

Official Code

Introduction

We address the task of zero-shot blind image super-resolution and propose a flow-based generative model, named as invertible kernel estimator (IKE) which aims to recover the high-resolution details from the low-resolution input image under a challenging problem setting of having no external training data, no prior assumption on the downsampling kernel, and no pre-training components used for estimating the downsampling kernel.

Algorithm

IKENet approximates the image downsampling and upsampling processes simultaneously via a flow-based generative model. Thanks to the invertibility of IKENet, the forward and backward flows are directly linked to the downsampling/kernel-estimating and upsampling/super-resolving steps. We provide the detailed algorithms of the forward and backward processes of the proposed IKENet with the following video.

Experiments and Results

DIV2k Track 2 Dataset (unknown degradation)

download link

The following table presents the super-resolved images on DIV2K dataset Track 2. The first four and the last four rows are the results of upsampling by 2 times and 4 times respectively.

Non-linear Degradation Dataset

The following table presents the super-resolved images on the non-linear degradation dataset. The first to the fourth rows are the results against Median, Bilateral, Anisotropic Diffusion and Random degradation kernels respectively.

RealSR Dataset

download link

RealSR is a dataset which consists of pairs of real-world HR and LR images and these paired images are captured with different focal lengths. The following table presents the super-resolved images on the RealSR dataset.

Ablation Study

We perform ablation study to analyze the contributions of each design in our model using DIV2k Track 2 dataset with scale factor x2, where the results are provided in the following table and the GIF animation. Both the tri-channel coupling layers for the IKE backbone and the bicubic residual design help to generate better SR images. Moreover, the contributions of different objective functions are discussed. Among them, $L_{energy}$ solves the color shifting problem and brings significant impact to our model. In addition, $L_{CSR}^{bwd}$ and $L_{inter}$ also play important roles for the performance improvement.

Reference

1. EDSR: Enhanced Deep Residual Networks for Single Image Super-Resolution [paper][code]
2. KernelGAN: Blind Super-Resolution Kernel Estimation using an Internal-GAN [paper] [code]
3. ZSSR: “Zero-Shot” Super-Resolution using Deep Internal Learning [paper] [code]
4. DualSR: DualSR: Zero-Shot Dual Learning for Real-World Super-Resolution [paper] [code]