IEEE Consumer Electronics Magazine - January/February 2020 - 96

Special Section on Cognitive Science and AI for Human Cognition and Communication

Consumer electronics (CE) data capturing sensors such as cameras nowadays are integrated
with communication electronics (e.g., smartphones). This integration accelerated the development and expanding of different CE fields to cover
almost all our life aspects. Implementing robust
gesture recognition algorithms as smartphone
applications will help hearing people understand
hearing-impaired people. Moreover, these algorithms could be part of complete systems that do
two ways translation between sign language and
spoken language. A successful translation system
will bridge the communication gap and facilitate
the hearing-impaired people life. The recognized
gestures, on the other hand, can also be used to
command CE devices such as TVs.
Like other time-variable signals, hand gestures
cannot be compared in a Euclidian space directly
because of the time dependence and its misalignments. This time dependence and the large irrelevant areas in each frame make it very difficult to
find real representative hand-engineered features
for hand gestures. To work well with conventional
classifiers, the required features should involve
robust descriptors for hand shape, position, orientation, and temporal dependence between consecutive frames. These features should also be
robust against different circumstances such as
occlusion and background clutter. For these reasons, deep learning is employed in this article,
since it seems to be a promising solution.
The CNN had been successfully utilized for
automatic feature learning. It achieved excellent
results in image classification, object recognition,1
and even in human activity recognition.2 This excellent performance is attributed to the availability of
huge databases, which contain millions of samples.
In contrast, the huge database requirement still not
satisfied in case of hand gestures. To overcome the
lack of labeled data for training deep models, in
this article, we investigate the transfer learning
principle. We present an efficiently adapted deep
model for automatic hand gesture recognition.
The main contributions in this article, are as
follows:
1. Presenting a 3D-CNN model for learning spatio-temporal features for hand gestures.
2. Adapting a pretrained deep model to work
well on the hand gesture recognition.

THE STATE OF THE ART
In past decades, significant studies have been
conducted on hand gesture recognition and
human-machine interaction in general. The
majority of studies have followed two main
approaches based on the data acquisition technique. In the first approach, the signer is accustomed to some interfacing devices such as data
gloves, position trackers, motion sensors, and
accelerometer to collect the hand gestures data.3
The drawback of this approach is its high cost
and inconvenience to the signer. The studies in
the second approach avoided these drawbacks.
The hand gestures in this approach recorded via
cameras and different imaging devices (no need
to contact the signer body and restrict his/her
movement).
Recently, a stereo matching was used by
Yang et al.4 for character formulation. The proposed method can be generalized for touchless
interaction, but the 3D calculations make it too
slow for real-time applications. A CNN-based
deep model was proposed to recognize seven
command gestures.5 The recognized gesture is a
static, even though it is presented in a frame
sequence. A block-based histogram of optical
flow representation for isolated SLR was proposed by Lim et al.6 The method gained accuracies of 93.33%, 60.0%, and 85.9%, respectively, on
three datasets, the RWTH-BOSTON-50, Purdue
RVL-SLLL ASL, and ASL Lexicon. A system based
on HMMs and discrete cosine transform was presented by Assaleh et al.7 The proposed system
achieved an accuracy of 94% on 23 isolated
words. K-NN and Bayesian classifiers were used8
to evaluate spatio-temporal representation technique of Arabic sign language. The best recognition rate of 100% was reported on the same
database.7 An SVM classifier for the local binary
pattern in three orthogonal planes was proposed
by Aly and Mohammed.9 An accuracy of 99.5%
was reported on the same dataset.7 A Wi-Fi
received-signal-strength-indicator (RSSI) recognition technology was proposed by Lao and
Wong.10 Statistical analysis techniques was used
on the RSSI values stored in the 802.11 radio
header to recognize the hand gesture. The performance of this method was affected by noise.
It is difficult to extend it to work with more complicated gestures.

IEEE Consumer Electronics Magazine

IEEE Consumer Electronics Magazine - January/February 2020

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - January/February 2020