IEEE Consumer Electronics Magazine - January/February 2020 - 98

Special Section on Cognitive Science and AI for Human Cognition and Communication

Figure 1. 3D-CNN model for feature learning and classification.

on the large-scale Sport-1M2 human action recognition dataset. The investigated model in this work
contains six blocks as illustrated in Figure 1. The
first block consists of a 3D-CNN layer (conv1) of 64
kernels followed by a max-pooling layer. The second block contains a 3D-CNN layer (conv2) of 128
kernels followed by a max-pooling layer. The third
and fourth blocks each include two successive
3D-CNN layers (conv A and conv B) and a maxpooling layer. The fifth block includes two successive 3D-CNN layers (conv A and conv B), a
zero-padding layer and a max-pooling layer. The
number of kernels in the 3D-CNN layers of these
three blocks are 256, 512, and 512, respectively.
The 3D-CNN kernels have a unified shape of
(3 Â 3 Â 3) through the whole system. The kernel
size of all pooling layers are (3 Â 3 Â 3) except the
one in the first block, which has a size of
(1 Â 2 Â 2). The final block contains two fully connected layers each of 4096 neurons and the output layer. In all hidden layers of the model,
rectified linear unit is used for activation, which
is the simplest nonlinear function to be used for
activation. We append a softmax prediction layer
to the last fully connected layer. The softmax activation function outputs the probability for each
of the classes. The predicted class is the one with
the highest probability. The described model
was fine-tuned by back-propagation with negative log-likelihood to address the hand gestures
recognition problem. The well-known stochastic
gradient descent optimizer (learning rate ¼10-4,

decay ¼ 10-6, and momentum ¼ 0.9) is used when
fitting the model on hand gesture datasets. The
stacked blocks of 3D CNN represent the spatiotemporal features of the input clip as 3D vectors
in different levels of abstraction.
The output of the last 3D-CNN layer was
flattened and forwarded to the fully connected
layers and then to the output layer. To achieve
good generalization for the model on unseen data
and avoid overfitting, we used dropout with a
probability of 0.5 after each fully connected layer.
The system was implemented in python and executed on a machine with Nvidia GTX 980 Ti GPU.

A CASE STUDY
The performance of the 3D-CNN-based system on the hand gestures is now studies using
the datasets described in Section III.

Experiments on KSU-SSL Dataset
Signer dependent mode: The selected part
from KSU-SSL dataset contains 8,000 video samples evenly distributed among 40 classes as
listed in Table 1. The dataset was split into training and test with ratios of 0.8 and 0.2, respectively. The training 6400 samples were used for
model fine-tuning over 100 epochs. At each
epoch, 5% of these samples were selected randomly for validation. The trained model was
then evaluated on the test dataset, which contains 1600 unseen samples. With a batch size of
16 samples, we got an accuracy of 96.69%.

IEEE Consumer Electronics Magazine

IEEE Consumer Electronics Magazine - January/February 2020

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - January/February 2020