IEEE Consumer Electronics Magazine - September/October 2020 - 95

The difficulty of the task is related to the
compression artifacts that can severely impact
the SR process. To our very best knowledge, we
are the first to propose such a compression
scheme, involving both state-of-the-art BPG compression and GAN SR networks.

SRGAN Model and Â2 Upsampling Scheme
In our work, we have modified the SRGAN network,13 which performs an upsampling by a Â2
factor. We have kept unchanged the original
SRGAN discriminator network. At the level of the
generator network, we followed the recommendation by Lim et al.15 in order to adjust our model.
For training purposes, the Mirflickr25k dataset
has been considered, which offers a fair variability in terms of both visual appearances and quality. A subset of 20k images has been used. This
volume proved to be sufficient for an efficient
training process, while being less demanding in
computational resources.
Recently, the so-called Enhanced SRGAN
(ESRGAN) extension of the SRGAN has been proposed.16 ESRGAN includes three different optimizations, recommended in Lim et al.15 First, the
network structure has been improved by modifying the core blocks in order to fit better to SR
applications. This new structure appears to be
more efficient and easier to train. Second, the
discriminator has been improved by using relativistic average (RaGAN) which does not judge,
among the presented images, "which one is fake"
but "which one is the more realistic." Finally the
perceptual loss is improved by using the VGG
features before activation and not after as it is
the case in the SRGAN.
As far as we know, ESRGAN represents today
the state-of-the-art SR technique.
ESRGAN Model and Â2 Upsampling Scheme
As in the case of SRGAN, a similar approach
has been performed with the ESRGAN network.
For this experiment, a different dataset was
needed. The ESRGAN requires fixed-size training
images at (480 Â 480) pixels. Authors recommend to use a dataset with HR images such as
DIV2K, and extract a set of subpatches to be
used as a training set. Following these recommendations, a low resolution dataset has been
created, including 44000 sub-images of 240 Â 240

September/October 2020

pixels, extracted from 1100 images from both
the DIV2K and Flickr2K datasets. The low resolution images have been BPG-compressed with the
same range of resulting bitrates than for the Â2
SRGAN-based experiment. We have also performed an additional training with an MSE loss
function in order to evaluate the differences
between the MSE and perceptual loss functions.

ESRGAN Model and Â4 Upsampling Scheme
Originally, both SRGAN and ESRGAN methods
perform upsampling with a factor 4. Moreover,
authors point out that better results can be
achieved if a pretraining with a peak signal to
noise ratio (PSNR) loss function is preliminary
performed. For this reason, a PSNR oriented pretrained model, with a factor Â4, is provided. In
our work, we have trained a Â4 reconstruction
adopting the same parameters as for the Â2 one.
In addition, we have included the PSNR pretraining stage. The low resolution corpus involves
the same images, downsampled here to 120 Â
120 pixels and very low bitrate encoded.

EXPERIMENTAL EVALUATION
In order to evaluate the proposed methodology, we have compared the different reconstructions yielded by our different models, on
various images with different visual content
and resolutions.
For evaluation purposes, we have considered
the 24 test images of the Kodak dataset, largely
used in the literature for benchmarking purposes.
In a first phase, we have performed a comparison between the three different approaches
introduced in the "Proposed Methodology" section (SRGAN Â 2, ESRGAN Â 2, and ESRGAN Â 4),
in order to determine which one offers the most
promising performances with respect to the considered application.

Comparative Evaluation
The criteria retained for this comparison concern the capacity of the methods to provide, at
very low bitrates, an overall pleasant image quality, blur, and artifact-free, while ensuring the
capacity of reproducing fine image details, such
as written text, facial features, or object patterns.

IEEE Consumer Electronics Magazine - September/October 2020

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - September/October 2020