IEEE Robotics & Automation Magazine - March 2022 - 42

useful for evaluating a multitude of social interactions (e.g.,
user pose for evaluating body language, user-robot proxemics,
and user gaze).
Although VR interfaces have the aforementioned
strengths for enhancing experimental evaluation, they have
their own set of unique evaluation challenges as well-one of
which is the use of online studies with crowdworkers (e.g., on
MTurk). HRI in general has made prolific use of online user
studies (especially during the COVID-19 pandemic) that
take advantage of cheap and readily available participants.
However, VAM-HRI heavily draws upon 3D visualizations
(as often seen with HMD-based interfaces), which cannot be
properly displayed to crowdworkers who lack HMDs and/or
3D monitors.
Additionally, a strength of AR interfaces is that 3D data
and visualizations can be rendered contextually in user environments
and are able to be observed from any angle desired
by the user. VAM-HRI studies that utilize crowdworkers to
evaluate VAM interfaces, such as those performed by Mott
et al. [18], are restricted to online images and videos viewed
by MTurk on 2D monitors that restrict the user's viewpoint to
that of prerecorded videos, which does not allow for a true
VAM experience. It remains an open question whether results
from crowdsourced VAM-HRI studies provide comparable
results to VAM-HRI studies run in person since 3D VAM
technology is inherently experienced differently than the 2D
experiences found on crowdsourcing platforms. Regardless,
using crowdworkers still holds value in the early prototyping
phases of VAM-HRI research where the initial formulation
of object and interaction designs can be evaluated quickly
and inexpensively.
VAM-HRI as an Interdisciplinary Study
HRI is well known to be an interdisciplinary field, and VAMHRI
is proving to be no exception. The CoBot Studio project
unites roboticists, psychologists, artificial intelligence experts,
multimodal communication researchers, VR developers, and
professionals in interaction and game design [14]. As the
VAM-HRI field grows, it will likely become increasingly common
(and needed) to see teams with varied experiences and
skill sets contributing to collaborative research.
Research in multirobot systems is an underexplored inspiration
for VAM-HRI research in regard to enhancing the CM.
VAM technology can be formulated as another robot within a
system-a robot with nondeterministic, nondirectly controllable
behavior but with a data-rich sensor suite. The frameworks
and techniques of the adjacent field may be able to be
modified or even directly applied when treating the human
user as an autonomous mobile sensor platform, akin to the
person being treated as though he/she is another robot in the
system. For example, spatial and semantic scene understanding
are important perceptual capabilities for both active
robots (to navigate their environments) and passive VAM
technologies (to localize the user's field of view).
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
Figure 3. Advances in VAM-HRI research have enhanced the ability to precisely record, play back, and analyze human interactions with
robots and other experimental stimuli in controlled user studies. This is exemplified in the CoBot Studio project of Mara et al. [14],
where HRI user studies are conducted in a VR environment with numerous virtual cameras monitoring the experimental area from a
multitude of angles. (a) Top, (b) side, (c) over the shoulder, (d) top angular, (e) side angular, (f) perspective, (g) user point of view,
and (h) robot point of view. These cameras make use of the VR hardware to track body and head motion to record human postures
and posture shifts, task-related human movements, gestures, gaze behaviors, and so on. Techniques such as these can benefit the
field of HRI as a whole and allow for more complete and feature-rich data of human behavior that would otherwise be lost without
VAM-HRI technology and recording techniques.
42 * IEEE ROBOTICS & AUTOMATION MAGAZINE * MARCH 2022

IEEE Robotics & Automation Magazine - March 2022

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - March 2022