Signal Processing - November 2017 - 138

impaired by one of five types of synthetic distortions: JPEG and
JPEG2000 (JP2K) compression, white Gaussian noise (WN),
GB, and Rayleigh fast-fading channel distortion. The differential
mean opinion score (DMOS) of each distorted image is provided.
The CSIQ database [32] includes 30 reference images and 866
synthetically distorted images of six types: JPEG, JP2K, WN,
GB, pink Gaussian noise, and global contrast decrements. The
DMOS of the distorted images is also provided. TID2013 [13]
contains the largest number of distorted images. It consists of 25
reference images and 3,000 synthetically distorted images with
24 different distortions at five levels of degradation. The database
also provides the mean opinion scores (MOS). The LIVE multiply distorted (MD) database [33] was the first to include multiple
(synthetically) distorted images. Images in it are distorted by two
types of distortion in two combinations: simulated GB followed
by JPEG compression and GB followed by additive WN. It contains 15 references and 405 distorted images, and the DMOS of
each distorted image is provided.
Finally, the LIVE Challenge database [3] contains nearly
1,200 unique image contents, captured by a wide variety of
mobile camera devices under highly diverse conditions. As
such, the images were subjected to numerous types of authentic
distortions during the capture process, often in complex combinations of multiple interacting impairments, as shown in Figure 5(b). The distortions include, e.g., low-light blur and noise,
motion blur, camera shake, overexposure, underexposure, a
variety of color errors, compression errors, and many combinations of these and other impairments. There are no reference
images in the LIVE Challenge database, since the distorted
images are originals, captured by dozens of ordinary photographers. The LIVE Challenge pictures were judged by more
than 8,100 human subjects in a tightly monitored -crowdsourced
study, yielding more than 350,000 human judgments that
exhibit excellent internal consistency [3]. A summary of the
attributes of these five databases is shown in Table 2.

Performances of CNN picture-quality models
Since only a few CNN-based picture-quality models have
been released, we provide the prediction accuracies of baseline
models on the five databases as performance references to be
compared against. We selected the well-known very deep CNN
models AlexNet [2] and ResNet50 [34] as the architectures of
the baseline models, where each was pretrained on the ImageNet

classification task. Both of these pretrained models are available
for download. The specific network configurations can be found
in the original papers. For each pretrained architecture, two
types of back-end training strategies were tested: using an SVR
to regress the extracted features from the CNN model onto subjective scores and fine-tuning the pretrained networks to conduct picture-quality prediction. We did not test direct training
of these models on any of the picture-quality databases, since
they are not large enough. Very deep networks easily overfit on
insufficient training samples, causing significant decreases in
testing accuracy (AlexNet has 62 million and ResNet50 has 26
million parameters). Instead, we tested a smaller CNN network
as a baseline model of direct training.
In the first approach, the output of the sixth fully connected
layer (4,096 dimensions) from AlexNet and averaged-pooled
features (2,048 dimensions) from ResNet50 were used as the
input feature vectors to the SVR. From each input image, 25
randomly cropped image patches (the patch size is predefined
by the pretrained models: 227 × 227 for AlexNet, and 224 ×
224 for ResNet50) were used for training and testing. The
obtained feature vectors from these 25 image patches were
averaged to obtain a single global feature vector.
In the second approach, we randomly cropped 100 image
patches from each training image to be used for training
(except on the TID2013 database, where 30 cropped patches
were used, due to the large number of distorted images in the
database). The image patches inherited the quality scores from
the source distorted images, which were first normalized to
the range [0, 1]. This preprocessing enabled us to use the same
parameter settings on all databases. The basic regression loss
(1) was used. To alleviate overfitting, one dropout layer with
dropout rate 0.5 was added before the last fully connected layer.
The learning rate was set to 10 -3, and the fine-tuning process
iterated for eight and six epochs on AlexNet and ResNet50,
respectively. The batch size was fixed at 48 for both models.
In the testing stage, the pretrained models were used to predict
quality scores on each of 25 random image crops. These were
average pooled to produce the final picture-quality scores.
For the direct training approach, we used the following
CNN architecture: Conv-48, Conv-48 with stride 2, Conv-64,
Conv-64 with stride 2, Conv-64, Conv-64, Conv-128, Conv-128,
FC-128, FC-128, and FC-1. Here, "Conv" refers to convolutional layers, "FC" refers to fully connected layers, and the trailing

Table 2. A comparison of IQA databases in terms of numbers of reference images, distorted images, distortion types,
authenticity of distortions, type of subjective scores, whether distortions are mixed, and published date.

138

Database

Number of
Reference
Images

Number of
Distorted Images

Number of
Distorted Types

Authenticity
of Distortions

Subjective
Score Type

Mixtures of
Distortions

Published
Date

LIVE IQA [12]

779

Synthetic

DMOS

N/A

2003

CSIQ [32]

866

Synthetic

DMOS

N/A

2010

TID2013 [13]

3,000

Synthetic

MOS

N/A

2015

LIVE MD [33]

405

Synthetic

DMOS



2012

LIVE Challenge [3]

N/A

1,162

Numerous

Authentic

MOS



2016

IEEE SIGNAL PROCESSING MAGAZINE

November 2017

Table of Contents for the Digital Edition of Signal Processing - November 2017