Instrumentation & Measurement Magazine 24-6 - 23

There has been research done in the field of ML for coaching.
In [11], researchers designed a Tai Chi coaching system
using an RGB-D camera and CNNs. This work is a valuable
contribution; however, it requires special equipment such as
the RGB-D camera, which is not common in traditional homes.
The research done in [12] is also related to Tai Chi, where the
authors used Random Forest and Support Vector Machines
(SVM) for motion estimation while using the expensive Cave
Automatic Virtual Environment (CAVE) environment for
feedback. Yoga has also attracted the attention of researchers.
For example, in [13], the authors used several infrared cameras
in conjunction with CNN for yoga posture recognition. Compared
to CAVE or infrared cameras, our approach does not
need complex sensor systems.
There is also work done for soccer, such as in [14], where
researchers focused on delivering a smart coaching system
directly to a smartphone, based on recommender systems to
prevent athlete injuries. The advantage of DTCoach, compared
to [14], is the automation of video processing using DL
pose estimation.
Since we are working on the edge, we have also examined
existing research work catered to mobile technology.
Such is the case of [15], where the authors used depth-wise
separable convolutions to reduce the number of calculations
done by the CNN. They also introduced width and
resolution hyperparameters to improve accuracy and
latency if needed. This architecture also gave birth to MobileNetV2
[16], where researchers tried to improve on
MobileNetV1 using linear bottlenecks and shortcut connections
between them.
System Design
Overview
This research aims to design a Digital Coaching System running
entirely on the edge, focusing on compatibility and
resource consumption. To achieve this, we designed and
developed DTCoach to run smoothly on the client-side with a
focus on the following points:
◗ Portability: to be able to have the system with us at all
times. Mobile devices have opened the door for EC as
users have access to more and more computing resources
at the palm of their hand.
◗ Compatibility: to be able to run the system anywhere,
anytime. The ubiquity of programming languages such
as JavaScript and HTML, along with DL frameworks
available for them such as Tensorflow.js, allow for the
development of robust AI applications that can run on
any device with modern browser support.
◗ Flexibility: To conduct measurements based on the
trainee and coach needs, we allow the user to select their
preferred routine video from widely available sources.
This takes minimal processing time since the pre-processing
of these routine videos can be done automatically
using DL pose estimation models.
◗ Adaptability: not all bodies are equal, and not everyone
has the same height and fitness level. Therefore, the
system should adapt its measurements to different people
seamlessly while providing enough relevant feedback for
every user based on their performance.
Fig. 1 shows the DTCoach workflow based on the DTCoach
ecosystem proposed in [20]. Here we have two environments:
the server side, which handles the coach activities in a robust
device; and the client side, which handles everything from the
trainee on the edge, therefore, with limited resources.
Coach Environment
The coach records the video of the routine that the trainee will
follow. This video will be then processed with a pre-trained
pose estimation model using ResNet50. We have the luxury
of using a heavy model since this will all be processed serverside
to access more computational resources than the trainee's
environment. The keypoints generated by the model will then
be saved into a file that will be accessed from the client-side.
Fig. 1. DTCoach Workflow.
September 2021
IEEE Instrumentation & Measurement Magazine
23
Instrumentation & Measurement Magazine 24-6

Table of Contents for the Digital Edition of Instrumentation & Measurement Magazine 24-6
