IEEE Robotics & Automation Magazine - September 2014 - 132

Temporal Signal of GSR
Rest

0.4
0.2

Level 2

0

-0.2
Level 1

-0.4
-0.6

0

100

200

300
Time (s)
(a)

400

500

Pulse Rate (beats/min)

GSR (nS)

0.6

Mean Pulse Rate Each 15 s

90

Fuzzy Output
GSR

0.8

Fuzzy Output
Pulse

85

Rest

80
75

Level 2

70
65

Level 1

60
55

0

100

200

300
Time (s)
(b)

400

500

Figure 10. The variation of the force control level on the basis of the output of the fuzzy logic system. (a) GSR and fuzzy logic system output.
(b) Pulse and fuzzy logic system output.

Figure 10 shows, for one of the tested subjects, the GSR
and mean pulse rate signal changes together with the fuzzy
logic system output in the case in which the force control gain
was varied, during the drinking task execution, from Level 1
(green) to Level 2 (pink); the resting phase (blue) is also
shown. In addition, the output of the fuzzy logic system,
which estimates the user's arousal and valence, is depicted
with a dotted black line; it was used to change the robot's control level and was computed every 30 s. Note that the pulse
signal, which is shown in Figure 10(b), has fluctuations
because of the way of computing it-the signal is computed as
the mean of the pulse signal every 15 s.
The reported tests intend to provide evidence of the feasibility of the proposed new concept of assistive device. Nevertheless, the number of successes in achieving the tasks over the
total number of trials also provides preliminary results on the
efficacy of the proposed overall system. Performance measurements during the drinking task, such as the number of successful drinking tasks during the experiments and the time to
properly perform the task, in ten healthy subjects are shown in
Table 3. The results show that the mean value of successful
drinking tasks decreases with the level of assistance provided
by the robotic device (from task 1 to task 3); on the contrary,
the mean value of the time taken to properly finish the task
increases with it.
The performance of the overall control system is difficult to
measure since the final goal of the system is to adapt the level of

Table 3. Performance measurements during the
drinking task for ten healthy subjects. The number
of successful drinking tasks during 5 min and the
time to properly perform the task is reported.
Drinking Tasks (Number)

132

*

Time (s)

Mean

Standard
Deviation

Standard
Mean Deviation

Task 1

14.43

4.04

15.43

6.38

Task 2

7.71

2.29

36.15

4.20

Task 3

8.00

2.00

25.75

1.05

IEEE ROBOTICS & AUTOMATION MAGAZINE

*

september 2014

task difficulty to the subject's biomechanical and emotional state.
However, the experiments carried out with ten healthy subjects
showed that the system is able to adapt to the subject's state by
changing the control level four or five times during the experiments. For a more in-depth evaluation of the performance of
such an assistive device, experimental trials with patients are
needed. This requires long periods of experimentation (around
three months) with the objective of involving clinical units and
measuring patients' outcomes with the clinical scales in addition
to the biomechanical and emotional evaluation performed here.
Conclusions
This article presents a new concept of multimodal assistive
robotic system with three main key features: 1) it is able to provide assistance as needed by implementing a user-tailored
approach that accounts for and adapts to subjects' residual
physical and cognitive abilities, 2) it can augment users' physical and cognitive abilities in performing daily living activities,
such as drinking, cooking, eating, personal hygiene, and
grooming, and 3) it provides an increased level of safety in the
interaction because of the specific arm design and the number
of sensors monitoring the user's state. The proposed assistive
device is composed of a 7-DOF modular robot arm conceived
for safe human-robot interaction, a multimodal HRI for physiological and biomechanical monitoring of the user, two communication channels for speech recognition and audio-visual
feedback, and the overall control system. It is able to manage
all of the acquired sensory information and provide the robot
with the commands needed for patient-tailored assistance.
Ten healthy subjects were tested for evaluating, on the one
hand, the reliability of the multimodal human-machine
interface and, on the other hand, the feasibility of overall system. The results confirm that it is possible to exploit users'
physiological data to estimate their emotional state and performance data to estimate their biomechanical state during
the execution of ADL tasks assisted by a robotic device. In
addition, the combination of GSR and pulse rate is a reliable
method for estimating the user's physical and cognitive
workload, while skin temperature and respiration rate could
be used depending on the subject.
Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2014
