IEEE Robotics & Automation Magazine - March 2022 - 66

user can initiate the handover task by saying a predefined
keyword. The ROS application will listen to the incoming
ROS message being published by the plugin to a predefined
topic and start the handover task as soon as the ROS message
is received by the ROS application.
The AR system was validated in a study with 16 participants
and their experiences were surveyed with a questionnaire
given at the end of the experiment. The results were
summarized in Figure 9(c). The AR systems received a higher
rating across three categories: task fluency, trust toward the
robot, and perceived safety. The results suggested that the use
of visualization and speech commands through ARviz both
greatly improves the experience of the users during the handovers,
and also leads to better efficiency in performing the task.
Potential Implementations and Limitations
Potential Implementations
The " Use Cases " section demonstrated the capabilities and
benefits of ARviz in a range of HRIs such as mobile robot
navigation, conveying motion intent and improving robot
handover experience. Recognizing the potential of ARviz, we
suggest two other robot applications that can be enabled by
our platform.
The first proposed application is an interactive interface for
a robotic manipulator using hand gestures. In this application,
ARviz could create a one-to-one virtual model of the robot,
with an interactive marker that would be attached at the robot's
end effector. By interacting with this marker, the operator can
move the virtual robot to any desired pose [similar to how the
MoveIt motion planner plugin for Rviz works, as shown in Figure
10(a)]. With the virtual robot model in AR, the operator
can preview directly in the robot's workspace the robot's motion
and ensure that there are no collisions with the surrounding
physical environment.
The second proposed application enables multihuman-
(a)
multirobot collaboration. ARviz was designed to be a standalone
AR software that can be deployed to any compatible
AR device. That means ARviz with the same configuration
settings, i.e., connecting to the same network, having the
same Display Plugins and Tool Plugins, can be deployed to
different AR devices and would be able to interact with the
same ROS applications at the same time. An example of this
would be two human operators each using ARviz on a HoloLens
2 collaborating with two mobile robots, as shown in
Figure 10(b). ARviz is deployed on both AR devices with
identical configurations, which connects to the central computer
that runs the ROS applications controlling the two
mobile robots. One operator can give navigation commands
as described in the " Mobile Robot Navigation " section.
The operator can choose which robot to operate
using a control panel for the navigation Tool Plugin or can
operate both at the same time. Their collaborator (or trainee)
can observe the task being implemented. The planned
path of the robots can also be visualized on both AR devices,
conveying motion intent of the robots to both human
collaborators.
(b)
Figure 10. The potential implementations with ARviz. (a) a MoveItinspired
interactive interface with the robotic manipulator, Franka
Emika Panda. The image shows the MoveIt-implemented interface
in ROS. (Source: MoveIt Tutorials [14].) (b) A visualization of a
potential ROS application that enables multihuman-multirobot
collaborations.
66 * IEEE ROBOTICS & AUTOMATION MAGAZINE * MARCH 2022
Limitations
While we have had early success in demonstrating the potential
of the platform in the " Use Cases " section, ARviz still has
a number of limitations requiring further development. Two
key development issues are discussed here.
The current implementation has limited accuracy in
aligning the virtual world and the real world. The semiautomatic
localization method with a QR code marker provides
the simplicity in implementation and usage, but the
IEEE Robotics & Automation Magazine - March 2022

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - March 2022
