IEEE Consumer Electronics Magazine - July 2018 - 18

PAC is designed according to the
method of game-based learning
and aimed at arousing a person's
interest in dance.
enhance generalization by reducing overfitting. Based on
this, we quantify movements of the choreographer and student by key skeleton-point information, including position,
direction, rotation, and velocity of the body. As the last step,
the motion pattern recognition can be constructed by classification of features selected from movements. Motion recognition and evaluation are often complementary to each other.
The method of movement recognition usually can be used for
movements evaluation, and the feedback of dance movement
is key to learning.

GAME-BASED DANCE LEARNING SYSTEM PAC
The schematic description of the overall system is shown in
Figure 3.

GAME-BASED DANCE LEARNING SYSTEM
The PAC needs to perform the same functions as a human
choreographer, including demonstration and feedback. The
standard movements demonstration is based on a standard
movements database that is installed with the professional
dancer's movements captured by the optical motion-capture
system. The movement data will be bound to the virtual
instructor model in the PAC system. The instructor driven by
the standard movement data can be shown on the screen for
educational purposes, and the feedback is made in real time
according to the comparison of the user's movement data with
the choreographer's standard movements database.
It is necessary to collect movement data of the choreographer's joints ahead of time, and it can be used as a reference

Get Movement Data
of User's Joints in
Real Time

Get Movement Data of
Choreographer's Joints
Ahead of Time

Encoding of
Skeleton
Vector Signals

july 2018

KEY TECHNIQUES
MOTION CAPTURE AND BINDING
Kinect is used to capture information on the 20 skeleton
points of the user in front of the camera, including x, y, and z
coordinate data for each point and the velocity, accelerated
velocity, and direction of each skeleton vector. For the role of
choreographer, standard waving dance movements were
recorded and encoded by an optical motion-capture system
according to the movement of a professional dancer ahead of
time to construct a standard movements database. The format
of movement is in Extensible Markup Language, and these
movements can be given to the model of the instructor to provide reference to students in the PAC. Students can learn
hand-waving dance movement directly in front of the Kinect
camera. The movements of the user can be recorded with
Kinect, and these encoded movements will be given to the virtual character model.
The 3-D motion data of the hand-waving dance after
motion capture and skeleton binding of the character are used
in the movement data binding process, which is completed in

User-toChoreographer
Alignment
Coordinate System
Alignment

Encoding of
Skeleton Vector
Signals to the
Construction of
Motion
Database

FIGURE 3. The schematic description of the overall system.

18 IEEE Consumer Electronics Magazine

to users. For learning, users can practice in front of the camera following the 3-D animation of the choreographer driven
by the standard movement data of the database. The movement data of the user's joints will be captured by Kinect in
real time, and the skeleton vector signals will be encoded for
evaluation. We must also align the movements of the user and
choreographer before comparison. Because the cameras used
for movement capturing might not be synchronized, and the
devices used for motion capture may be different, the coordinate system, key points of the skeleton, and entering point of
the movement must be aligned before evaluation. At the same
time, weightings made according to the features of the dance
can promote the reliability of evaluation results. Ranking the
scores of each body part and visualizing the evaluation
results can help users adjust their movements and master the
dance more effectively.

User-withChoreographer
Comparison
Setting Weight
According to the
Feature of Dance

Key Points of
Skeleton Alignment

Ranking of Each Part
Score

Entering Movement
Alignment

Visualization of
Evaluation Results
Movement Data Processing

Evaluation
Results

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - July 2018