Signal Processing - November 2017 - 139

were taken from the original papers. Overall, the CNN-based
full-reference models followed by the CNN-based no-reference models achieved higher prediction accuracies relative to
conventional learning-based models on the legacy databases.
Table 3 compares the performance of the various picturequality prediction models on all of the reviewed databases. The
last five rows show results for the baseline models. The three best
performing no-reference picture-quality models in each column
are boldfaced. Generally, the existing CNN-based models were
able to achieve remarkable prediction accuracies on the legacy
databases. However, it is much harder to obtain successful results
on the LIVE Challenge database. For example, the model proposed in [27], DIQA, achieved an SRCC of 0.687, which is lower
than the results attained by a recent successful SVR-based method, FRIQUEE-ALL [21], which achieved an SRCC of 0.72.
However, the baseline models that were pretrained on the
ImageNet databases achieved standout accuracies on the LIVE
Challenge database. This is likely because the real-world ImageNet pictures are not synthetically distorted. Instead, like the
LIVE Challenge pictures, any distortions occurred as a natural
consequence of photography, without intervention by the database creator. This further suggests that the pretrained CNNs are,
to some degree, already quality-aware, meaning that their learned
internal features assist the performance of the task (recognition)
by adapting to the presence of authentic distortions.
The baseline models using the first approach achieved very
low accuracies on the legacy databases, since they were not
exposed to any synthetic distortions during training, and hence
the learned features were not very useful to the SVR for quality prediction. Fine-tuning the pretrained baseline deep models
significantly improved performance on the legacy synthetic databases, but not enough to make them competitive, since there was
not enough data to train them adequately. The exception was the
directly trained shallow CNN baseline model, which achieved
competitive performance on the legacy databases, but lower
accuracies on the LIVE Challenge database.

numbers indicate the number of feature maps (of Conv) or output nodes (of FC). The model accepts 112 × 112 images as
inputs. All of the convolutional layers were configured to use
3 × 3 filters, using zero-padding to preserve the spatial size.
Each layer used a rectified linear unit as the activation function.
Following the convolutional layers, each 28 × 28 feature map
(assuming two convolutional layers with a stride of two) was
averaged yielding an 128-dimensional feature vector, which is
then fed into the fully connected layers. The number of parameters in this model is about 0.4 million, which is much lower
than AlexNet or ResNet50. This baseline model was trained
using the imagewise L 2 loss in (3). Each input image was
partitioned into 112 × 112 patches when training on the LIVE
IQA database, while full-sized images were used on the other
databases. On the LIVE IQA database, nonoverlapping patches
were used so that overlapped regions could not be accessed
multiple times by the CNN model during training and/or testing. The data was augmented by supplementing the training set
with horizontally flipped replicas of each image. Each minibatch contained patches extracted from five images. The training was iterated over 80 epochs.
Two performance metrics were used to benchmark the models: Spearman's rank order correlation coefficient (SRCC), and
Pearson's linear correlation coefficient (PLCC). To evaluate the
baseline models, we randomly divided each database into two
subsets of nonoverlapping content (distorted or otherwise), 80%
for training and 20% for testing. Of course, all of the LIVE Challenge pictures contain different contents. The SRCC and PLCC
were averaged after ten repetitions of this random process.
The performances of all of the exemplar picture-quality
prediction models on the LIVE IQA database are shown in
Figure 6. The first five (from left) are no-reference learningbased models, where the last two of these used deep learning.
The next seven are CNN-based no-reference-quality prediction
models, and the last three are CNN-based full-reference models. The reported SRCC and PLCC scores of the listed models

No-Reference Models

0.99
SRCC
PLCC

0.98
Correlation Coefficient

Full-Reference Models

0.97
0.96
0.95
0.94
0.93
0.92
0.91

M
[2
ee
6]
pQ
A
[2
7]

[2
5]

N
N
C

ee
p
D

-D
FR

4]
A

[2
3]
IQ
D

O
N
EC

gh
te

[2
2]

2]
[2

1]
e

[2
hw

N
tc
Pa

ee
p

[2
0]

[1
9]

[1
6]
SA

[1
8]
Q

A-

IQ
U

-A

N
R
FR

]
[8

E
U
Q
IS

O
C

[1
0]

0.9

FIGURE 6. A comparison of the SRCC and PLCC of learned picture-quality models on the legacy LIVE IQA database.
IEEE SIGNAL PROCESSING MAGAZINE

November 2017

139

Table of Contents for the Digital Edition of Signal Processing - November 2017