IEEE Consumer Electronics Magazine - January/February 2020 - 99

Signer independent mode: In this experiment,
we trained the model on samples performed by
some signers and evaluated it on samples performed by other signers. 6,400 samples performed by 32 signers were used for fine-tuning
the model. The trained model was then evaluated
on 1,600 samples produced by eight signers. With
a batch size of 16 samples, we got an accuracy of
72.32%. Figure 2 shows the system performance.

Experiments on ArSL Dataset
The actual valid samples of ArSL dataset that
were used in our experiments were exactly
3,444 samples.
Signer dependent mode: In this experiment,
nearly two-thirds of the dataset (2,298 samples)
were randomly picked for system fine-tuning and
the last third (1,146 samples) for evaluation. For
validation, 5% of the 2,298 samples were randomly
selected at each epoch. With a batch size of
16 samples, an accuracy of 100% was achieved.
Signer independent mode: In this experiment,
the 2,298 gestures' samples performed by the first
two signers were used for system training. With
the same batch size of 16 samples, the system
accuracy on test data was not more than 34.9%.
This low accuracy was not a surprise in signer
independent mode. There are only two signers'
samples seen by the system before testing by the
third signer's samples. The system did not see
enough samples with reasonable variation to be
generalized well. In contrast, the system achieved
better accuracy for signer independent mode on
KSU-SSL. There are 32 signers in the training dataset, which means better generalization.
Experiments on Purdue RVL-SLLL ASL Dataset
The dataset involved in our experiments contains ten gestures performed by 14 signers with
two repetitions each. That means the total number
of samples used is only 280. With such limited data
size, it will be difficult to train and generalize the
model. For that reason, in the second part of this
experiment, we investigated the effect of data augmentation on the system performance.
Part 1 without data augmentation-Signer dependent mode: After shuffling the dataset, we randomly picked 80% of the whole dataset (224
samples) for training and the rest 56 for test. It
achieved a recognition accuracy of 62.5%.

January/February 2020

Figure 2. Confusion matrix on KSU-SSL signer independent
mode.

Signer independent mode: In this case, we used
all the samples performed by 11 signers for training. The total number of training samples is 220.
The remaining 60 samples, performed by three
other signers, were used for evaluation. The system achieved a recognition accuracy of 58.33%.
Part 2 with data augmentation: In this part, we
split the dataset into 220 samples for training
and 60 samples for test as in the last part. The
training samples had been augmented before
being used to train the model. To achieve that,
we performed rotation and Gaussian blurring. A
rotation operation with small angles of (3 , 6 ,
9 , and 12 ) was performed on each training
sample to produce extra four samples. Three
Gaussian blurring operations of kernel size (3,
5, and 7) were also applied to each training
sample to produce extra three samples. The
overall augmentation process ended up with a
training dataset of 1670 samples.
Signer dependent mode: For this mode, the original training 220 samples were selected randomly
from all signers. Then, augmented to 1,670 samples
and finally used to fit the model with the same
setup used in the previous experiments. The
remaining 60 samples from the original dataset
was used to test the trained system. The recognition accuracy was raised up from 62.5% to 76.67%
by data augmentation (see Figure 3).
Signer independent mode: For this mode, the
original training 220 samples were produced by

IEEE Consumer Electronics Magazine - January/February 2020

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - January/February 2020