IEEE Consumer Electronics Magazine - July 2018 - 17

should be given by a dance coach and will be imitated by
learners in the demonstration-performance method.
Meanwhile, the instant feedback from the choreographer is
key to effectively learning to dance. This learning relies mainly
on the choreographer watching and analyzing students' movements and fixing their nonstandard movements to improve
their performance. The choreographer is expected to give a
demonstration and feedback. This method is common and easy
to learn, but students are not allowed to control the process of
learning. Generally, the number of trainees is much larger than
that of trainers, which makes it hard for the choreographer to
pay close attention to each student. It is significant to be able to
provide a personal choreographer for each learner so that the
students can control the pace according to their individual
learning process. While this dance movement learning method
has gained a lot of attention, few studies have combined it with
technology. To fill the gap, we mainly focused on designing
and developing the PAC for the hand-waving dance to demonstrate and give feedback to a student's dance performance.

GAME-BASED LEARNING
Game-based learning has attracted the attention of many
researchers because it can inspire a person's interest through
interactive entertainment. Some research has focused on the
reasons why game-based learning has resulted in widespread
public interest and how it improves students' learning efficiency on a theoretical level [9]. Beilock and Goldin-Meadow demonstrate how body movement can influence a
learner's cognition and that the motor system is connected to
the mind system [10]. Game-based learning on motor skills
training accounts for the highest percentage among related
studies [11]. On a practical level, many studies have proved
the effectiveness of game-based practice and the design and
application principles of games used in instruction [12].
Game-based learning promotes learning not only for its
motivation of interest but also for its support of powerful
strategies for that learning, such as the authentic environment, situated learning, and optimized challenge and scaffold [13]. Given the importance of amusement in learning,
PAC is designed according to the method of game-based
learning and aimed at arousing a person's interest in dance.

Depth Image

Inferred Body Parts

Game-based learning promotes
learning not only for its motivation
of interest but also for its support
of powerful strategies for
that learning.
TECHNOLOGY-ENHANCED MOVEMENT
RECOGNITION AND EVALUATION
Movement recognition is the first step of quantitative dance
study during which a computer tests movement data and symbolizes its information. Recognizing movement is a complex
process that involves moving-target detection, feature extraction, and motion pattern recognition. There are two kinds of
motion-capture systems in choreography, an optical motioncapture system, in which passive or active markers are placed
at the body joints, and a motion sensor device, in which movements can be captured by depth and red-green-blue information. Optical motion-capture systems are complex and
expensive but accurate [14]. In the optical motion-capture system, the choreographer's key point of his or her body is
marked by optical transmitters, and the cameras of the system
are used to record movement data. Kinect is low cost and convenient but accompanied with noise. Because choreographers'
movements will be imitated by students, the motion-capture
system should be reliable and exact. For students, it is too
complex to wear the markers for learning each time. Considering the feasibility and usability, an optical motion-capture system is used to record the professional choreographer's dance
movement data, which will be stored in the database. The
Kinect is used to capture movements of users. Standard movement matching is the main method for evaluation in PAC. Skeletal tracking technology is the core of it and can recognize 20
key points of the body accurately and track them in real time.
The working principle of Kinect is illustrated in our previous
research [15] and is shown in Figure 2.
In the second step, a feature extraction technique is used
to simplify models to make them easier for interpretation and

Hypothesized Joints

Tracked Skeleton

FIGURE 2. The concept diagram for Kinect motion capture.

july 2018

IEEE Consumer Electronics Magazine

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - July 2018