IEEE Robotics & Automation Magazine - September 2018 - 14

the pilot station for teleoperation and inspection purposes. To
cope with the variety of light conditions in postearthquake
scenarios, the robot head
is equipped with four
light-emitting diode units
The communication
(brightness 690 lm/unit,
channel plays a paramount power 6 W/unit). Their
strobing and light intensity can be actively conrole for the achievement
trolled by the pilot to
tune them according to
of our objectives.
need. The robot is powered by a custom lithium-ion battery (29 V-63 Ah) that provides it with about 3 h
of power autonomy.
Pilot Station and Teleoperation Interfaces
The WALK-MAN-pilot interface (PI) [13] is used by the
operator to send high-level commands to the robot and visualize its kinematic state, which is displayed in the 3-D environment surrounding it (Figure 5). A monocular scene image
is also visualized in the interface.
A custom human-machine interface (HMI) has been
realized to teleoperate the robot [Figure 4(a)-(c)]. The HMI
is composed of an immersive 3-D viewer and four inertial
and electromyographic bracelet sensors to control the movement of the robot arms and hands. The Myo bracelets [30]
are used to acquire the teleoperator's electromyography
(EMG) and inertia measurements. We decided to place one
Myo bracelet on the forearm and one on the bicep of the
pilot. A Madgwick filtering algorithm [14] is used to obtain

the orientation of each Myo. Hence, the relative orientation
between the two devices is used to calculate the wrist pose
given the length of the pilot's arms. Finally, a linear combination of electromyographic signals from the forearms are processed, as reported in [15], to extract a signal used as a reference
for the control of the robot's hand closure. This method also
allowed us to cope with the issues of placement and repeatability of EMG sensors, because each operator follows a short
training session (1 or 2 min) to obtain a mapping from the
EMG signals to hand closure signals. More information
about the use of EMG sensors for controlling the Pisa/IIT
SoftHand can be found in [16]. Virtual-reality viewer Oculus
Rift [31] has been used to exploit human stereo vision and
reproduce 3-D scenes, and its inertial unit and infrared sensors have been used to estimate its pose in the space. The stereo images coming from the ZED camera are sent to the 3-D
viewer for a visual feedback from the robot. The orientation
of the teleoperator's head, used for robot gaze teleoperation,
is computed using the inertial sensor placed in the Oculus
system. The teleoperator's wrist pose and level of hand
closure is sent to the control module that translates the information into control inputs for the robot joints (see the "Teleoperation Module" section).
On the communication side, the main personal computer
(PC1) was directly connected to the Ethernet cable dedicated
to the commands sent by the teleoperator, while the second
cable was connected to a router that also establishes a local
network between all of the pilot PCs through an Ethernet connection. In this way, the teleoperator receives the visual data in
the Oculus Rift while sending his or her head orientation,
wrist pose, and hand-closure references to PC1. Finally, the

Figure 5. The PI used by the operator of PC1. The 3-D viewer is used to understand the scene and take measurements.

IEEE ROBOTICS & AUTOMATION MAGAZINE

september 2018

IEEE Robotics & Automation Magazine - September 2018

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2018