Signal Processing - November 2017 - 122

in image statistics, these features lead to improved results for
object and action classification for different data sets.
The authors in [70] study the features extracted from the
final layers of a deep neural network for a fixed set of object
classification and detection tasks. The generic features from the
fifth, sixth, and seventh fully connected layers of an AlexNet
[43] show remarkable adaptation properties and outperform
state-of-the-art methods in classification and detection. Whereas [70] studied adaptation using CNNs, [71] studied adaptation
of features extracted using stacked denoising autoencoders for
text-based sentiment classification.

Insights
To boost the performance for a data set using a shallow procedure, naïve methods can be applied using deep networks as feature extractors [69], [70]. Sometimes, there may be constraints
to deploy only shallow methods due to data set size, hardware
resources, etc. In such situations, naïve methods can be very
effective. When domain discrepancy between the source and
the target is not very large, pretrained deep networks provide
highly adaptive features for the source and target.

Adopted shallow methods
These sets of deep-learning methods adopt shallow (nondeep
learning) domain adaptation procedures in a deep neural network. In these approaches, the features extracted by the layers of the deep network are learned to be domain invariant.
Domain alignment is achieved either through MMD, moment
alignment [64], or a loss function that drives the source and
target classifiers to be indistinguishable. In discussing these
methods, the central idea is outlined, leaving out the details
of network architecture, optimization procedures, loss functions, etc.
In [72], the authors adapt an AlexNet [43] to output domain
invariant features in the final, fully connected fc8 layer in the

deep domain confusion algorithm. The network has two loss
components: 1) softmax classification loss for the source data
points and 2) domain confusion loss. The network minimizes
an MMD loss over the source and target data outputs of the fc8
layer in every minibatch during training. This is termed the
domain confusion loss. A related idea is studied in [73], where
the network has a domain confusion loss along with a domain
classification loss. The domain classification loss ensures the
output feature representations of the source and target data are
distinct. This is in contrast to the goal of the domain confusion
loss, which tries to learn domain-invariant representations.
The network is trained to alternately minimize the two losses
and reach an equilibrium. Both of these methods assume the
presence of a few labeled samples in the target domain.
Long et al. introduce the deep adaptation networks (DAN)
model [63], which extends the concept of domain confusion by
incorporating an MMD loss for all of the fully connected layers
( fc6, fc7, and fc8) of the AlexNet. The MMD loss is estimated
for the feature representations over every minibatch during training. The work also introduces MMD estimation computed with
an efficient linear complexity based on [74]. The linear MMD
estimation is also unbiased because the MMD for the entire
source and target data can be expressed as the sum of MMD
across minibatches. Based on the network architecture of the
DAN [63], Venkateswara et al. [65] develop a hashing algorithm
for domain adaptation. The architecture of the domain adaptive
hash (DAH) network is based on the VGG-F, and domain alignment is achieved using MMD just like in the DAN. Figure 1
depicts the architecture of the DAH, which is similar to the
DAN. The network is trained in an iterative manner using
batches comprising source and target data. The loss function of the DAH has three components: 1) MMD loss for the
fully connected layers fc6, fc7, and hash-fc8, which aligns the
features of the two domains; 2) supervised hash loss for the
source, which extracts unique hash codes for every category

Convolution Layers

Source
Data

MK-MMD

Target
Data
conv1

conv2

conv3

conv4

conv5

fc6

fc7

hash-fc8

Entrophy Loss

Hash Loss

Fully Connected Layers

FIGURE 1. The DAH [65] network outputs hash codes of d dimensions for the source and target images. The architecture of the DAH is similar to the

domain adaptation network [63]. Adaptive features are extracted by using an MMD loss between the source and target data points in each subset for the
fully connected layers fc6, fc7, and hash-fc8. fc: fully connected. conv: convolution. (Figure used courtesy of [65].)

122

IEEE SIGNAL PROCESSING MAGAZINE

November 2017

Table of Contents for the Digital Edition of Signal Processing - November 2017