IEEE Robotics & Automation Magazine - March 2022 - 77

Consequently, a new model extension is proposed, and its
applicability is validated across all the experimental results,
showing improved modeling and representation of human
performance in combined movements of 3D object pointing
and manipulation tasks (Figure 1). Finally, the implications of
3D interaction, teleoperation, and object task design in VR
are discussed.
Beyond the Gold Standard of Fitts's Law
With recent advances in networking and mixed reality (MR)
technologies and in conjunction with the increasingly more
immersive applications of telepresence and teleoperation, the
need to measure and model human performance in 3D space
has exponentially increased [1], [2]. However, a compelling
standardized metric for 3D object selection, such as those
seen in VEs and teleoperation, does not exist. The absence of
a standardized metric severely limits interstudy comparability
and, most importantly, transferability between results, due to
the multitude of different measurements researchers can use
[3]. Consequently, progress toward the endeavor of a standardized
formulation is still " scattered " and often disregards
important aspects that would support a concrete and established
metric [4].
To propose a higher-dimensional metric for assessing
human performance, we investigate Paul Fitts's original predictive
model, short for Fitts's law [5], [6]. Proposed in 1954,
the law has been extensively used in human-computer interaction
(HCI) and ergonomics research and still represents the
gold standard as a performance metric [2], [7]. This is attributed
to the advantage of using Fitts's law to measure human
performance in a time-based approach based on spatial data,
effectively combining time and spatial units under one formulation.
Fitts's law was originally formulated for 1D translational
movements [5], but it has been extended to 2D tasks
[7], [8], with its applicability highlighted in rotational tasks, as
well [9]-[11]. Recently, Fitts's formulation was extended to
some extent in 3D space, limited to translational tasks, with
numerous reformulations [1], [12], [13]. Yet, these studies
either disregarded important spatial aspects or had findings
limited to specific settings. This aggravates interstudy comparisons,
particularly due to variations in tested motor tasks.
More specifically, previously proposed 3D metrics extending
Fitts's law have disregarded combining translational and
rotational tasks under one setting [1], [12], [13] and assessing
directional [10], [11] and inclination variations [1], [10],
[11], [13] in one exhaustive study. All the aforementioned
factors appear to have significant effects on human performance
[2], [4]. More importantly, to date, most studies
focused only on pointing tasks, and thus there is a severely
limited focus on manipulation tasks, e.g., with the incorporation
of grasping and physical properties, such as gravity
and friction forces. Consequently, we studied the factors and
spatial complexities seen in full 3D space to investigate their
contribution toward a human performance model. More
specifically, we performed an exhaustive user study in PPS,
i.e., close to a user's body, in a VE with a VR headset and
MARCH 2022 * IEEE ROBOTICS & AUTOMATION MAGAZINE *
77
az
θ = 90°
VR Headset
ϕ = 180°
Optical Hand
Tracker
ϕ = 270°
ϕ = 90°
θ = 0°
ϕ = 0°
A
ay
Virtual Cube
Object
Fz
Fx
Fy
ω
Wx
Wy
Virtual Cube
Target
ax
ω
Wz
ω
Shadow
Dexterous Hand
Figure 1. An operator interacts with objects in full 3D VR with all task-related spatial variables.
IEEE Robotics & Automation Magazine - March 2022

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - March 2022
