Signal Processing - November 2017 - 80

deep framework. The network has a symmetrical structure,
where two subnetworks were connected by a cosine similarity
layer. There are two convolutional layers and a full connected
layer for each subnetwork. Their method has two key advantages: 1) it can learn a s- imilarity metric from image pixels directly;
2) it can learn multichannel filters to capture both the color and
texture information from body images simultaneously.
Most DML methods assume that the training and testing
samples are collected in similar scenarios and the same distribution assumption is usually made. This assumption does not
hold in many real world applications, especially when samples
are captured across different data sets. To address this, Hu et al.
[23] proposed a deep transfer metric learning (DTML) method
to learn hierarchical nonlinear transformations for cross-domain
visual recognition, which learned transferrable discriminative
knowledge from the labeled source domain to the unlabeled target domain. Specifically, DTML learned a deep metric network
by maximizing the interclass variations and minimizing the
intraclass variations, and minimizing the distribution divergence
between the source domain and the target domain at the top layer
of the network. To better exploit the discriminative information
from the source domain, they also considered exploiting discriminative information from the middle layers of the deep network so
that more discriminative information can be exploited.
Recently, Lu et al. [14] introduced a multimanifold DML
(MMDML) method to recognize objects form -d ifferent
view--points or under different illuminations. Specifically,
MMDML jointly learns multiple nonlinear feed-forward neural networks, one for each object class, to explicitly project the
instances from each image set into a common feature space
at the top layer of all networks, where the maximal manifold margin constraint is enforced. In this way, class-specific
discriminative information can be effectively exploited for
classification. The authors' method achieved competitive performance on five widely used image set data sets.
Table 1 shows basic characteristics of several Siamese
networks-based DML methods. In this table, the strongly
supervised setting means that the class labels of t- raining data
are used to train neural networks, and the weakly supervised
setting denotes that only the pairwise labels of -similar pairs
and dissimilar pairs are used to train neural n- etworks.
Table 1. Characteristics of several DML methods using
Siamese networks.

Method

Setting

End to End?

Convolutional
Architecture?

DrLIM [19]

Strongly supervised

Yes

DNLML-ISA [20]

Weakly supervised

DDML [6]

Weakly supervised

DeepFace [21]

Strongly supervised

Yes

DeepID2 [7]

Strongly supervised

Yes

DTML [23]

Strongly supervised

MMDML [14]

Strongly supervised

DML via triplet networks
DML using triplet networks was trained by minimizing a triplet
loss function, which exploits labels of training data to generate
+
triplets. Given a triplet (x i, x +
i , x i ), x i is a similar example to the
reference x i, and x i is a dissimilar example to the x i . A triplet
+
(x i, x +
i , x i ) means that x i is more similar to x i in contrast to x i ,
+
i.e., d f (x i, x i ) 1 d f (x i, x i ). DML via triplet networks aims to
minimize the following loss function for triplets:
L ^{W (m), b (m)} mM= 1 h = / i h ^x + d f (x i, x i+) - d f (x i, x i-)h, (5)
where h (x) = max (0, x) is the hinge loss function, and x 2 0 is
a margin between d f (x i, x +
i ) and d f (x i, x i ). The triplet network
pulls the similar example close to reference and pushes dissimilar
example further away.
Wang et al. [9] proposed a deep ranking model with the triplet-based hinge loss functions to learn similarity metric from raw
images. Specifically, they employed a multiscale neural network
architecture to capture both the global visual properties and the
image semantics. An efficient online triplet sampling method
was presented to generate a large amount of training data to learn
the parameters of the network.
Hoffer et al. [26] employed a triplet network architecture for
DML, which aims to learn useful representations by distance
comparisons. Their method is similar to the approach in [9] that
learned a deep ranking similarity -function for image retrieval.
Their method made a comprehensive study of the triplet architecture, and demonstrated that the triplet approach is a strong competitor to the Siamese approach.
Schroff et al. [24] introduced a FaceNet deep model that
directly learns a mapping from the original sample space to a
compact Euclidean space. Once this space is produced, face recognition and clustering can be easily implemented under the network. Specifically, FaceNet used a deep convolutional network to
directly optimize the embedding itself rather than using an intermediate bottleneck layer. Triplets of roughly aligned matching/
nonmatching face patches were generated for training with an
online triplet mining method.
Bell and Bala [27] proposed learning visual similarity for
product design with the CNNs, which exploit communities of
users to help each other answering questions about products in
images. Their method contains two different domains of product images: products cropped from internet scenes, and products in their iconic form. With the help of a multidomain deep
embedding, it can deal with several applications of visual search
including identifying products in scenes and finding stylistically
similar products.
Song et al. [28] introduced a DML -method via lifted structured feature embedding (LiftedStruct) to learn se-----mantic feature embeddings where similar examples are mapped close to
each other and dissimilar examples are mapped farther apart.
Their method took full advantage of the training batches in
the network training stage by lifting the vector of pairwise
distances within the batch to the matrix of pairwise distances. This step enabled the method to learn the state of the art
feature embedding by optimizing a new structured prediction

IEEE SIGNAL PROCESSING MAGAZINE

November 2017

Table of Contents for the Digital Edition of Signal Processing - November 2017