IEEE Robotics & Automation Magazine - March 2022 - 59

source code for our ARviz platform is available at: https://
github.com/hri-group/arviz.
Background and Motivation
ROS [1] has risen to become the framework of choice for
robotics research and development. ROS is used in many
robotic research areas, ranging from mobile robots and robotic
manipulators to aerial robotics, as well as applications ranging
from factory and health care robots to disaster response
and space exploration robots. The industrial-compatible version
ROS2 is under active development.
When working with robots, the ability for users to visualize
sensor data, understand robot state, and issue commands
in an intuitive way is crucial. The successful software tool,
RViz (http://wiki.ros.org/rviz), offers these capabilities for
the development of robotics applications built on the ROS
framework through a 2D, screen-based visual interface
[Figure 1(b)]. However, providing visualizations and interfaces
in a 2D screen for a 3D world has inevitable challenges
with respect to intuitiveness (especially with respect to spatial
understanding) for the user. Furthermore, to see the visualization/information
or to command the robot, a screen-based
interface requires the user to frequently shift their attention to
the monitor screen, disconnecting them from the physical
robot/environment. This attention shifting results in reduced
levels of safety and efficiency when operating or collaborating
with robots.
AR technology offers the capacity to create and superimpose
virtual objects and information directly onto the real
world, as viewed by the operator, colocated in the robot's
space and task space. In recent years, AR technology has
risen in popularity becoming a promising new alternative for
creating interfaces that can facilitate more intuitive and effective
interaction and information exchange in human-robot
collaborations [2]. AR-based interfaces offer a richer visual to
the user and more intuitive interaction through virtual
objects superimposed onto the real world. By rendering relevant
information in the user's field of view directly over the
real world, AR interfaces help reduce mental fatigue and cognitive
load of the user when collaborating with their robot
partner [3].
Recognizing the potential of AR technology, researchers
have proposed various AR-based interfaces to assist HRI over
the past few years. Yew et al. [4] developed an AR-based teleoperation
interface for a maintenance robot, where a mixture
of real and virtual objects was used to reconstruct the remote
maintenance site at the operator's environment. Makhataeva
et al. [5] increased a human operator's awareness of danger in
human-robot collaboration by augmenting their views
through an AR display with a safety aura around the operating
robot. Walker et al. [6] used an AR display to show various
visual cues such as arrows and points to convey motion
intent of aerial robots during collaboration between human
and robot. Dinh et al. [7] proposed a novel interface for collaborating
with a tape dispensing robot. Their system allowed
the operator to monitor the task through an AR headset, and
enhanced the user's experience working with the robot.
Waymouth et al. [8] developed an interaction design for
demonstrating cloth folding to robots through AR. Chan
et al. [9] utilized AR in conjunction with gestures and tactile
feedback to create a multimodal system for robot programming
and execution,
where the AR display was
used to assist the human
operator with robot programming
in addition to
visualizing robot trajectories.
A similar system was
later on applied toward
large-scale human-robot
collaborative manufacturing
tasks [10].
The number of ARARviz
is built as a platform
incorporating a collection
of plugins that provide
visualization and/or
interaction components.
HRI research papers in
recent years increased rapidly from only 4,000 papers in 2015
to 8,000 papers in 2019 [11]. However, most of these AR
applications developed are task-specific (including works
mentioned earlier), lacking generalizability, reusability, and
scalability; i.e., the AR-HRI applications developed for these
tasks are not easily repurposed for other tasks. To enable more
efficient application development and research on ARenabled
robotics and HRI, a universal platform that enables a
systematic approach to implementing data visualization and
(a)
(b)
Figure 1. Display Plugin used for visualizing tf (http://wiki.ros
.org/tf) ROS message data type on our platform ARviz (a) and on
Rviz in ROS (b). The similarities between the two visualization
tools makes using ARviz more convenient.
MARCH 2022 * IEEE ROBOTICS & AUTOMATION MAGAZINE *
59
https://github.com/hri-group/arviz https://github.com/hri-group/arviz http://wiki.ros.org/rviz http://www.wiki.ros.org/tf http://www.wiki.ros.org/tf
IEEE Robotics & Automation Magazine - March 2022

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - March 2022
