IEEE Systems, Man and Cybernetics Magazine - October 2023 - 47

achieve the multimodal synchronization and decoding of
proportional wrist/hand kinematics, passing offline data
tests and online data tests and obtaining better performance
than sEMG. Although AUS signals can capture deep
muscle characteristics [21], [24], their principle is equivalent
to scanning muscles according to a certain frequency,
resulting in a lack of temporal information.
Based on previous research, hand gesture recognition
using AUS signals has made notable progress in static gesture
recognition [25]. In 2022, a method for dynamic hand
gesture recognition using wearable AUS was introduced
for the first time, called the dynamic time warping (DTW)
algorithm [26]. The DTW algorithm employs numerical distance
to match similar samples, providing an effective
solution for dynamic gesture recognition. However, DTW
relies on traditional feature extraction, which has limitations
in extracting signal attributes. Deep learning, with
its robust learning ability, adaptability, data-driven nature,
and portability, can more comprehensively extract signal
features. Our aim is to present a dynamic hand gesture
recognition algorithm based on AUS signals using a deep
learning framework.
This article proposes and evaluates a deep learning
algorithm for dynamic hand gesture recognition using AUS
signals. By implementing the LSTM framework, we imbued
the AUS signal with temporal correlation and successfully
achieved dynamic gesture recognition. The experiment
involved handwritten digit recognition for " 0 " through " 9, "
which will be used to create datasets for future research
on dynamic AUS signal-based hand gesture recognition.
Through experiments, the accuracy of different deep
learning structures (CNN and LSTM) and traditional feature
engineering with an SVM classifier on the dynamic
gesture recognition of US signals is compared. We prove
that LSTM has better performance and that the recognition
accuracy of the whole dataset reaches 89.5%. It paves
the way for potential HCI applications involving dynamic
hand gestures.
The main contributions of our article are shown
as follows.
◆ We designed and collected a new dynamic hand gesture
dataset based on AUS signals.
◆ We introduced a dynamic gesture recognition
approach utilizing wearable AUS signals. By incorporating
temporal information into the AUS signal
through the use of LSTM, we broadened the research
scope of AUS signals and established a theoretical
foundation for potential future applications.
◆ We compared different algorithms (LSTM, CNN, and
SVM) on dynamic hand gesture recognition of US signals
and selected a better-performing algorithm.
The rest of this article is organized as follows. The
" Data Acquisition and Preprocessing " section introduces
data acquisition and preprocessing. The " Methodology "
section describes the proposed methods. The
" Dataset " section presents the datasets and parameters
(a)
(b)
of our experiments. The " Result " section introduces the
results and analysis of different groups, and the " Discussion "
section discusses all of the experimental
results. The last section provides a conclusion and outlook
for this study.
Data Acquisition and Preprocessing
In this article, we utilize a four-channel AUS signal acquisition
instrument designed and manufactured by Hangzhou
ELONXI Company as our signal acquisition equipment.
The four AUS signal probes are attached to the surface of
the forearm using elastic straps, while the computer collects
the raw AUS signals through the HCI system. Figure 1
illustrates the four-channel US equipment, an HCI interface
demonstration, categories of handwritten numerals,
and an example of a four-channel AUS signal. The communication
mode between the equipment and the computer is
Ethernet. During the acquisition process, US coupling
agents should be applied between the probe and the skin
to minimize the influence of air and the external environment,
thereby ensuring signal quality.
The US equipment consists of four channels, each corresponding
to an individual US probe. The sampling frequency
of the US equipment is 20 MHz, and the working
frequency of the US probes is 2.25 MHz. Each channel
has 1,000 sampling points. The software's timer operates
at a frequency of 20 Hz, meaning that it collects 20 frames
of data from the US device per second. Each frame contains
4,000 points, forming a
41 ,000)
matrix. To eliminate
the influence of invalid information from both the
surface and deep layers of the skin, 20 points from the
beginning and end data of each frame and channel are
removed during signal processing, leaving only the middle
960 points. Consequently, the actual matrix becomes
a 4 960) matrix.
(c)
(d)
Figure 1. (a) The ELONXI US equipment. (b) A demonstration
of the system setup. (c) The handwritten
numeral task. (d) Four-channel AUS signal visualization.
October 2023 IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE 47
IEEE Systems, Man and Cybernetics Magazine - October 2023

Table of Contents for the Digital Edition of IEEE Systems, Man and Cybernetics Magazine - October 2023
