Signal Processing - November 2017 - 59

nant classes is attributed to the large volumes of classificasubspace. It is conjectured that the existence of universal pertion regions corresponding to dominant labels in the input
turbations fooling classifiers for most natural images is partspace X: in fact, images sampled uniformly at random from
ly due to the existence of such a low-dimensional subspace
that captures the correlations among different regions of the
the Euclidean sphere aS d -1 of the input space X (where the
decision boundary. In fact, this subspace "collects" normals
radius a is set to reflect the typical norm of natural imagto the decision boundary in different regions, and perturbaes) are classified as one of these dominant labels. Hence,
tions belonging to this subspace are therefore likely to fool
such dominant labels represent high-volume "oceans" in the
other data points. This observation implies that the decision
image space; universal perturbations therefore tend to fool
boundaries created by deep neural networks are not suffiimages into such target labels, as these generally result in
ciently "diverse," despite the very large number of parameters
smaller fooling perturbations. It should be noted that these
in modern deep neural networks.
dominant labels are classifier specific and are not a result of
A more thorough analysis is provided in [30], where unithe visual properties of the images in the class.
versal perturbations are shown to be tightly related to the curTo further understand the geometrical properties of classivature of the decision boundary in the vicinity of data points.
fication regions, we note that, just like natural images, random
Specifically, the existence of universal
images are strongly vulnerable to adversarperturbations is attributed to the existence
ial perturbations. That is, the norm of the
The study of robustness
of common directions where the decision
smallest adversarial perturbation needed to
allows us to derive insights change the label of a random image (samboundary is positively curved in the vicinabout the classifiers and,
ity of most natural images. Figure 10 intuipled from X ) is several orders of magnitude
tively illustrates the link between positive
smaller than the norm of the image itself.
more precisely, about
curvature and vulnerability to perturbaThis observation suggests that classification
the geometry of the
tions; the required perturbation to change
regions are "hollow" and that most of their
classification function
the label (along a fixed direction v) of the
mass occurs at the boundaries. In [28], furacting on the highclassifier is smaller if the decision boundther topological properties of classification
dimensional input space.
ary is positively curved, than if the deciregions are observed; in particular, these
sion boundary is flat (or negatively curved).
regions are shown empirically to be conWith this geometric perspective, universal perturbations cornected. In other words, each classification region in the input
respond exactly to directions where the decision boundary is
space X is made up of a single connected (possibly complex)
positively curved in the vicinity of most natural images. As
region, rather than several disconnected regions.
shown in [30], this geometric explanation of universal perWe have discussed in this section that the properties and
turbations suggests a new algorithm to compute such perturoptimization methods derived to analyze the robustness
bations as well as to explain several properties, such as the
properties of classifiers allow us to derive insights on the
diversity and transferability of universal perturbations.
geometry of the classifier. In particular, through visualizations, we have seen that the decision boundaries on normal
random sections have very low curvature, while being very
Classification regions
curved along a few directions of the input space. Moreover,
The robustness of classifiers is not only related to the geomthe high vulnerability of state-of-the-art deep networks to
etry of the decision boundary, but it is also strongly tied to
universal perturbations suggests that the decision boundthe classification regions in the input space X. The classifiaries of such networks do not have sufficient diversity. To
cation region associated to class c ! {1, f, C} corresponds
improve the robustness to such perturbations, it is therefore
to the set of points x ! X such that f (x) = c. The study of
key to "diversify" the decision boundaries of the network
universal perturbations in [22] has shown the existence of
and leverage the large number of parameters that define the
dominant labels, with universal perturbations mostly fooling
neural network.
natural images into such labels. The existence of such domi-

∗
radv

(a)

(b)

(c)

FIGURE 10. The link between robustness and curvature of the decision boundary. When the decision boundary is (a) positively curved, small universal
perturbations are more likely to fool the classifier. (b) and (c) illustrate the case of a flat and negatively curved decision boundary, respectively.
IEEE SIGNAL PROCESSING MAGAZINE

November 2017

Table of Contents for the Digital Edition of Signal Processing - November 2017