IEEE Signal Processing Magazine - January 2018 - 68

Face? : Yes
CNN
Input Image

Extract Region
Proposals (~2 k/Image)

Compute CNN Features

Classify Regions
(Linear SVM)

(a)

Face
Detection
Scores at
Each Location
CNN
Deep Pyramid DPM

Input Image
Image Pyramid

Nonface

(b)

FIGURE 1. A typical DCNN-based face detector using the (a) region-proposal approach and (b) the sliding-window approach.

at low-resolution stages, and only evaluates a small number of
challenging candidates at high-resolution stages.

Single-shot detector
Liu et al. [8] proposed the single-shot detector (SSD) for the
task of object detection. The SSD is a sliding-window-based
detector, but instead of creating image pyramids at different scales, it utilizes the inbuilt pyramid structure present in
DCNNs. Features are pooled from the intermediate layers of
DCNN at different scales that are used for object classification and bounding-box regression. Because the detections are
obtained in a single forward pass of the network, the overall
computation time of SSD is lower than faster R-CNN [19].
Few SSD-based face detectors have been proposed recently.
Yang et al. proposed ScaleFace [28] to detect faces of different
scales from multiple layers of the network and fuse them in
the end. Zhang et al. proposed S3FD [29], which uses a scaleequitable framework and scale compensation anchor-matching
strategy for improved detection of small faces. Figure 1 shows
the architectures for the aforementioned approaches.
The availability of large training data sets for unconstrained
face detection has also contributed significantly to the success
of DCNN-based face detectors. The FDDB [30] data set is the
most widely used benchmark for unconstrained face detection.
It consists of 2,845 images containing a total of 5,171 faces collected from news articles on yahoo.com. The MALF [31] data
set consists of 5,250 high-resolution images containing a total
of 11,931 faces. The images were collected from Flickr and an
image search service provided by Baidu, Inc. These data sets
contain faces in unconstrained settings with large variations in
occlusion, pose, and illumination.
68

The WIDER Face [18] data set contains a total of 32,203
images, with 50% samples for training and 10% for validation. This data set contains faces with large variations in pose,
illumination, occlusion, and scale. Face detectors trained on
this data set have achieved improved performance [19], [23],
[28], [29], [32], [33]. The evaluation results for this data set
reveal that finding small faces in the crowd is still challenging. A recent method proposed by Hu et al. [33] shows that
contextual information is necessary for detecting tiny faces.
It captures semantics from lower-level features and context
from higher-level features of a DCNN to detect tiny faces (see
Figure 2). Due to space limitations in this article, we are unable
to discuss more traditional face detection methods. We refer the
interested readers to [34] for more detailed discussions of traditional cascade-based methods and deformable part-based model
(DPM) methods. In addition, for videos with multiple faces, one
also needs to perform face associations to extract face tracks of
each subject. We also refer the readers to [12] for a discussion
on the role of face association in video-based face recognition.
Figure 3(a) provides a comparative performance evaluation of
different face detectors on the FDDB [30] data set.

Finding crucial facial keypoints and head orientation
Facial keypoint detection is also an important preprocessing
component for face recognition and verification tasks. Facial
keypoints such as eye centers, nose tip, mouth corners, etc. can
be used to align the face into canonical coordinates, and such
face normalization helps face recognition [35] and attribute
detection. Head-pose estimation is required for pose-based
face analysis. Both of these problems have been researched
extensively over the last few years. Most of the existing facial

IEEE SIGNAL PROCESSING MAGAZINE

January 2018

http://www.yahoo.com

Table of Contents for the Digital Edition of IEEE Signal Processing Magazine - January 2018