IEEE Robotics & Automation Magazine - March 2022 - 67

tradeoff is the spatial accuracy of the overlaid visual elements,
as they are affected by the accuracy of the placement
of the QR code marker. An improved localization algorithm
is needed in future versions of the platform.
ARviz is also limited in its ability to change the topics that
plugins connect to at run-time. The ROS topics that ROS# collects
data from can't be modified during runtime and must be
preconfigured prior to the deployment of the platform onto the
AR device. This connectivity limitation for plugins is inconvenient
for AR-based ROS applications where multiple data
sources are involved-for example, an ROS application that
needs to visualize data collected from multiple cameras mounted
at different places on the robot. Future versions of ARviz will
allow users to modify ROS topics in run-time for one plugin, so
that the visualization of multiple data sources with the same
ROS message type can be achieved. A workaround for the current
version would be to create multiple instances of the plugin,
with each connecting to a different topic.
Conclusion
In this work, we introduced the ARviz platform, an open
source platform for AR interaction with ROS-based robotic
systems. ARviz supports visualization of various standardized
data types that are available on ROS using Display Plugins.
We also introduced the implementation of Tool Plugins that
utilizes the natural input methods supported by the AR headset
such as speech and gesture, to communicate with the ROS
environment. ARviz provides an easy-to-access and extendable
hand menu to control the plugins, allowing future developers
to expand the platform. We present three different use
cases of the platform from our labs, from conveying mobile
robot motion intent to visualizing hand over tasks. As demonstrated
in published user studies conducted using the
ARvis system, the platform provides a number of benefits,
with users feeling safer when collaborating with the robot and
completing the given task more efficiently.
Finally, we proposed two different potential applications
that can highlight the benefits of ARviz: an interactive
" MoveIt " type interface for a robotic manipulator and an application
that enables multihuman-multirobot collaboration.
Despite the success of the platform, there is still a great deal of
room for improvement, including improving the accuracy of
the semiautomatic virtual world alignment and addressing
limitations in connecting plugins to topics at run-time. We
continue to work on this exciting platform to improve capabilities
and to bring a better experience to end users.
References
[1] M. Quigley et al., " Ros: An open-source robot operating system, " in Proc.
ICRA Workshop Open Source Softw., Kobe, Japan, 2009, vol. 3, no. 3.2, p. 5.
[2] S. A. Green, M. Billinghurst, X. Chen, and J. G. Chase, " Human robot
collaboration: An augmented reality approach-a literature review
and analysis, " in Proc. Int. Design Eng. Techn. Conf. Comput. Inf. Eng.
Conf., 2007, pp. 117-126, doi: 10.1115/DETC2007-34227.
[3] S. Brending, A. M. Khan, M. Lawo, M. Müller, and P. Zeising,
" Reducing anxiety while interacting with industrial robots, " in Proc.
2016 ACM Int. Symp. Wearable Comput., New York, NY, USA: Association
for Computing Machinery, pp. 54-55, doi: 10.1145/2971763.2971780.
[4] A. Yew, S. Ong, and A. Nee, " Immersive augmented reality environment
for the teleoperation of maintenance robots, " Procedia Cirp, vol.
61, pp. 305-310, Apr. 2017, doi: 10.1016/j.procir.2016.11.183.
[5] Z. Makhataeva, A. Zhakatayev, and H. A. Varol, " Safety aura visualization
for variable impedance actuated robots, " in Proc. IEEE/SICE
Int. Symp. Syst. Integration (SII), 2019, pp. 805-810, doi: 10.1109/
SII.2019.8700332.
[6] M. Walker, H. Hedayati, J. Lee, and D. Szafir, " Communicating robot
motion intent with augmented reality, " in Proc. ACM/IEEE Int. Conf.
Human-Robot Interaction, 2018, pp. 316-324, doi: 10.1145/3171221.
3171253.
[7] H. Dinh, Q. Yuan, I. Vietcheslav, and G. Seet, " Augmented reality
interface for taping robot, " in Proc. 2017 18th Int. Conf. Adv. Robot.
(ICAR), pp. 275-280, doi: 10.1109/ICAR.2017.8023530.
[8] B. Waymouth et al., " Demonstrating cloth folding to robots: Design
and evaluation of a 2D and a 3D user interface, " 2021, arXiv:2104.02968.
[9] W. P. Chan, C. P. Quintero, M. Pan, M. Sakr, H. Van der Loos, and E.
Croft, " A multimodal system using augmented reality, gestures, and
tactile feedback for robot trajectory programming and execution, " in
Proc. ICRA Workshop Robot. Virtual Reality, Brisbane, Australia, 2018,
pp. 21-25.
[10] W. P. Chan, G. Hanks, M. Sakr, T. Zuo, H. Machiel Van der Loos,
and E. Croft, " An augmented reality human-robot physical collaboration
interface design for shared, large-scale, labour-intensive manufacturing
tasks, " in Proc. 2020 IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS),
pp. 11,308-11,313, doi: 10.1109/IROS45743.2020.9341119.
[11] Z. Makhataeva and H. A. Varol, " Augmented reality for robotics: A
review, " Robotics, vol. 9, no. 2, p. 21, 2020, doi: 10.3390/robotics9020021.
[12] M. Gu, A. Cosgun, W. Chan, T. Drummond, and E. Croft, " Seeing
thru walls: Visualizing mobile robots in augmented reality, " in Proc.
IEEE Int. Symp. Robot Human Interactive Commun. (RO-MAN), pp.
406-411, 2021, doi: 10.1109/RO-MAN50785.2021.9515322.
[13] R. Newbury, A. Cosgun, T. Crowley-Davis, W. P. Chan, T. Drummond,
and E. Croft, " Visualizing robot intent for object handovers with
augmented reality, " 2021, arXiv:2103.04055.
[14] D. T. Coleman, I. A. Sucan, S. Chitta, and N. Correll, " Reducing the
barrier to entry of complex robotic software: A moveit! case study, "
JOSER - J. Softw. Eng. Robot., vol. 5, pp. 3-16, May 2014.
Khoa C. Hoang, Monash University, Clayton VIC 3800,
Australia. Email: khoahoang08091997@hotmail.com.
Wesley P. Chan, Monash University, Clayton VIC 3800,
Australia. Email: wesleypchan@gmail.com.
Steven Lay, Monash University, Clayton VIC 3800,
Australia. Email: arbiter1000@gmail.com.
Akansel Cosgun, Monash University, Clayton VIC 3800,
Australia. Email: akansel.cosgun@gmail.com.
Elizabeth A. Croft, Monash University, Clayton VIC 3800,
Australia. Email: elizabeth.croft@monash.edu.
MARCH 2022 * IEEE ROBOTICS & AUTOMATION MAGAZINE *
67
IEEE Robotics & Automation Magazine - March 2022

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - March 2022
