IEEE Systems, Man and Cybernetics Magazine - October 2020 - 60

Participants
Fifteen healthy subjects were recruited to test and evaluate
the system (see Figure 3). In total, four females (26%) and
11 males (74%) participated in the experiment. The average
age range of the subjects was 26.6
[standard deviation (SD)=10.5]. The
participants' experience with video
The results indicate
games, as measured by a scale
ranging from 0 (no experience) to
that the system
10 (very experienced) was, on
was perceived as
average, 6.2 (SD=2.305).

Warnings (Compensation Alarm)
To avoid compensatory strategies from the patient, any
compensatory movement (e.g., bending the back) is
ignored by the system and does not help the patient reach
the virtual objects.
Rewards
The reward menu allows the patients
to spend the coins they earned
through their gameplay. The patients
are presented with a matrix of
cubes, each of which hides one collectible object. To unlock the objects,
the patients have to play a slot
machine game, where they can earn
coins or a collectible.

usable, regardless of
gender or previous
experience with
video games.

The Therapist's Interface
The current work focuses on the
patient's interface. Hence, the therapist's interface was
developed to provide the basic functionalities necessary to
support the proper functioning of the patient's interface.
The functions currently implemented are add/update
patient's data, adjust in-game parameters, and create/
update game sessions.
Experiment Design
Objective
An experimental design was developed to evaluate the
usability of the system, the impact of two design approaches (i.e., A1 and A2) on the perception of fun and difficulty
by users and how the developed system affects the motivation of the users.

Figure 3. A participant using the system.
60

IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE O ctober 2020

Procedure
The experiment consisted of one
session per participant. The playing
time of each game was fixed to 1
min, but the participants could play
the same game as many times as
they wanted. The participants were
instructed to test all the features of the system, but the
order for testing them was left to their preference. After
testing all the features of the system, the participants were
asked to complete a questionnaire to collect data about
their perceptions of the system.
Measurements
To obtain subjective evaluations of the system, a posttest
questionnaire was presented to participants to gain their
feedback on the following:
◆ system usability: A good level of system usability must
be guaranteed to avoid negatively affecting the user's
motivation, engagement, and adherence. The system
usability scale (SUS) was employed to measure the
usability of the system as a whole [23].
◆ difficulty and fun: A set of questions were designed for
measuring the participants' perception of difficulty
and fun for each game. The scale ranges from 0 to 10,
where "0" means that the system is very easy and "10"
denotes that it is very hard to use. For the aspect of
fun, "0" means that the system is very boring and "10"
is that it provides a high level of fun.
◆ motivation: To determine the system's effect on the participants' motivation, the situational motivation scale (SIMS)
[24] was used to measure their intrinsic motivation, identified regulation, external regulation, and amotivation.
Data Analysis and Results
To test the collected data for normality, a Shapiro-Wilk
normality statistical test was performed. Then, the oneway analysis of variance was used to analyze normally distributed data, and a nonparametric Kruskal-Wallis test
was used to analyze data that are not normally distributed.
This statistical procedure was used in the study conducted
by van Beek et al. [25]. The data related to the age of the
participants was rather homogenous; hence, no analysis
was made regarding the range of age.

IEEE Systems, Man and Cybernetics Magazine - October 2020

Table of Contents for the Digital Edition of IEEE Systems, Man and Cybernetics Magazine - October 2020