IEEE Robotics & Automation Magazine - March 2022 - 78

optical hand tracking. The VE consisted of simulated teleoperation
tasks with high-fidelity physics leveraging an
anthropomorphic, dexterous robotic hand. By collectively
adding 3D spatial variables in four progressively more complex
experiments, we derived our own model extension and
compared the proposed metric with the state of the art while
also verifying their applicability at each stage.
From our results, to the best of our knowledge, we present
a first-of-its-kind 3D human performance metric based
on Fitts's law, extending beyond current work by modeling
full 3D space through a superior approach. More specifically,
our metric is able to capture 3D human motion entailing
combined translational and rotational movements, with
varying degrees of directions and inclinations in object
pointing and manipulation under a single formulation. This
metric can be used to assess human performance by modeling
the complex motions, degrees of freedom (DoF), and
dimensions associated with VEs that include VR [1], [14] as
well as teleoperation [3], [15], [16]. Consequently, the effects
of different user interfaces, devices, and robotic systems on
user performance can be modeled and assessed, with the
added advantage of combining time and spatial metrics in
one representation. A video presentation is available at
https://youtu.be/MKH2gGJQq2o.
The contributions of our work are summarized as follows:
●
●
We propose a new higher-dimensional metric to assess
human motor performance in full 3D space.
We introduce an intuitive motion retargeting of hands in
high-fidelity physics to enable the generalization of our
method toward virtual and realistic object interaction and
manipulation.
●
A thorough comparison of our proposed metric with others
in the literature is provided under different experimental
settings, and the measurement's validity for higher
dimensions is shown.
●
The article includes a study of quantities and variables to
better explain the 3D spatial relationship of objects, and it
presents recommendations for the design of pointing and
manipulation tasks in VR and teleoperation.
Related Work
In this section, we introduce Fitts's original law for translational
tasks and the current state of the art of its 2D and 3D
extensions as well as the importance of including rotation
in a formulation. Ultimately, we investigate the incorporation
of translational and rotational movements into one
model. In Table 1, we summarize all the model extensions.
Fitts's Original Formulation
Fitts's formulation has been extensively used in motion prediction
in HCI and ergonomics research [5], [6]. The formulation
predicts the movement time (MT) for how long it takes
a user to point to a target on a screen. It is formulated as
MT ab ID ,.2 log
=+ $
where ID =
`
W
A
2
j
(1)
Here, A represents the distance between an object and a target,
W is the width of the target area, the logarithmic term ID
shows the task's index of difficulty measured in bits per second,
and the resultant MT is measured in seconds. The constants
a and b represent the y-intercept and slope, respectively,
and are derived via regression analysis.
Importance and Limitations of Fitts's Law
While extensively used, the formulation in (1) suffers in terms
of simplicity when full 3D space is considered. More specifically,
the formulation is limited to four key areas, namely, 1) lowerdimensional
1D and 2D space; 2) fewer DoF entailing only
translational tasks without the combination of rotational movements;
3) pointing tasks without the addition of physical properties,
such as gravity and friction, e.g., the manipulation of
objects; and 4) single-line movements without the use of spatial
arrangements, e.g., directions and inclinations. During interactions
in VR and teleoperation, all four aspects are a fundamental
and inseparable part of human motion in 3D [2], [10], [14].
Nevertheless, the ability of the law to combine time and
spatial metrics under a single formulation renders the pursuit
of extending it to 3D of significant importance. As identified,
part of the motivation of this article stems from supplementing
Table 1. A summary of the most widely used models based on Fitts's law and those compared with our model.
Performance
Model
Model Formulation and Equation
MT (s)
Fitts [5]
MT ab ID$=+
Hoffmann [8] MT ab ID$=+
Welford [17] MT ab ID$=+
Shannon [7] MT ab ID$=+
MT ab ID$=+
Murata and
Iwase [13]
Cha and
Myung [12]
MT ab cd ID12
=+ $$ $ii
sin
ID (bit/s)
=
2
2
ID log AW22
^
/
h
ID log AW Fh=+^2 2 /
ID lo /.g AW 05h=+^
ID log AW 1h=+^
/
ID=+ i
lo /sin
g ^2 AW c1h+ $
++ ID log AW Fh=+^2 2 /
N/A
[5]
[5]
[5]
[7]
2D
2D
2D
2D
3D
[8], [13] 3D
Model Characteristics
Derived
From Space Directions Inclines Rotation
No
No
No
No
Yes
Yes
Each model's characteristics are summarized, and the table indicates whether 3D spatial variables are considered.
No
No
No
No
No
Yes
Yes
No
No
Yes
No
No
78 * IEEE ROBOTICS & AUTOMATION MAGAZINE * MARCH 2022
https://www.youtube.com/watch?v=MKH2gGJQq2o
IEEE Robotics & Automation Magazine - March 2022

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - March 2022
