Signal Processing - July 2017 - 33

Three-Dimensional Shape Correspondence Application
Finding intrinsic correspondence between deformable
shapes is a classical tough problem that underlies a broad
range of vision and graphics applications, including texture mapping, animation, editing, and scene understanding [107]. From the machine-learning standpoint,
correspondence can be thought of as a classification
problem, where each point on the query shape is
assigned to one of the points on a reference shape (serving as a label space) [108]. It is possible to learn the correspondence with a deep intrinsic network applied to
some input feature vector f (x) at each point x of the
query shape X, producing an output U H (f (x)) (y), which is
interpreted as the conditional probability p (y | x) of x
being mapped to y [Figure S7(a)]. Using a training set of
points with their ground-truth correspondence {x i, y i} i ! I,
supervised learning is performed minimizing the multinomial regression loss

UΘ

xi

X

min H

/ log U H (f (x i)) (y i)

i!I

(S14)

with respect to the network parameters H. The loss penalizes for the deviation of the predicted correspondence
from the ground truth. We note that, while producing
impressive results [Figure S7(b)], such an approach
essentially learns pointwise correspondence, which then
has to be postprocessed to satisfy certain properties,
such as smoothness or bijectivity. Correspondence is an
example of structured output, where the output of the network at one point depends on the output in other points
(in the simplest setting, correspondence should be
smooth, i.e., the output at nearby points should be similar) Litany et al. [109] proposed intrinsic structured prediction of shape correspondence by integrating a layer
computing functional correspondence [106] into the deep
neural network.

yi

Y
(b)

(a)

FIGURE S7. (a) The learning shape correspondence: an intrinsic deep network U H is applied pointwise to some input features defined at each point.

The output of the network at each point x of the query shape X is a probability distribution of the reference shape Y that can be thought of as a
soft correspondence. (b) The intrinsic correspondence established between human shapes using intrinsic deep architecture (MoNet [54] with three
convolutional layers). Signature of histogram orientations (SHOT) descriptors capturing the local normal vector orientations [110] were used in
this example as input features. The correspondence is visualized by transferring texture from the leftmost reference shape. For additional examples,
see [54].

the graph. Furthermore, note that the graphs at each layer of
the network need not be the same. Thus, we can replace the
original neighborhood structure with one's favorite multiscale
coarsening of the input graph and operate on that to obtain the
same flow of information as with the convolutional nets above
(or rather more like a locally connected network [81]). This
also allows producing a single output for the whole graph (for
translation-invariant tasks), rather than a per-vertex output, by
connecting each vertex to a special output node. Alternatively,
one can allow h to use not only Wf and Df at each node but
also W s f for several diffusion scales s > 1 (as in [45]), giving
the GNN the ability to learn algorithms like the power method
and more directly accessing spectral properties of the graph.
The GNN model can be further generalized to replicate other
operators on graphs. For instance, the pointwise nonlinearity h

can depend on the vertex type, allowing extremely rich architectures [50], [51], [55], [79], [80].

Charting-based methods
We now consider the second subclass of non-Euclidean learning problems, where we are given multiple domains. A prototypical application the reader should have in mind throughout
this section is the problem of finding correspondence between
shapes, modeled as manifolds (see "Three-Dimensional
Shape Correspondence Application"). As we have seen, defining convolution in the spectral domain has an inherent drawback of the inability to adapt the model across different
domains. We will therefore need to resort to an alternative
generalization of the convolution in the spatial domain that
does not suffer from this drawback.

IEEE SIGNAL PROCESSING MAGAZINE

|

July 2017

|

33
Table of Contents for the Digital Edition of Signal Processing - July 2017
