IEEE Robotics & Automation Magazine - March 2022 - 17

animations and dynamic markers, as
shown in Figure 6(k), were designed
to guide the user through the procedures
used to pressurize and depressurize
the malleable link. Once the
user is happy with his or her positioning
of the distal link with the virtual
distal link, he or she can lock it
in place, completing the malleable
robot's reconfiguration [Figure 6(l)].
Performance Evaluation of the
AR Application
Workbench
Microsoft HoloLens
Computation of
Desired
Malleable Robot
Topology
Computation of
Workspace
Computation of
Desired EndEffector
Location
Initial Alignment Accuracy
We first evaluate the accuracy of manually
aligning the holographic environment
to the real environment's
origin, located at the malleable robot's
P1 (see Figure 2). Six OptiTrack Flex3
cameras surround the robot, with
reflective tracking markers located on
the points P1-5, defining the malleable
robot's topology [6]. The calibration
report of the cameras detailed a
mean 3D error for overall projection
of 0.426 mm. The holographic environment
was then manually aligned to
the real environment by a user dragging
the robot's base hologram onto
the real robot's base. Once satisfied with the alignment, the
holographic environment's position was locked.
The initial alignment error is difficult to measure directConfirm
Configuration
Reject
Configuration
HoloLens
Feedback
Confirm
Hologram
Positioning
Save
Scene
Reconfiguration
Guidance
Exit and
Reload
Figure 7. The reconfiguration workflow achieved by the AR-HRI system.
ly via optical tracking, but it can be inferred using three
randomly distributed subpoints within the achievable P5
theoretical space, defined by the P5 theoretical space when
Y > 0 (see Figure 4), which is visually bounded by the
reconfiguration space and mathematically described using
(1)-(4). As the relative positions between the points and
the robot's base are constant across the real and virtual
spaces, an offset between the virtual and real points can
only result from a misalignment in the robot's base. From
the virtual space, the location of the holographic points P ,Hi
v
with respect to the virtual origin was provided as a vector
,
v =+ + Hi
Hi Hi
,, ,,
Hi
v
Px iy jz k where i 12= ,, and 3, denoting
the three subpoints, respectively. An arm with a tracked
marker at the end of it was then placed in the real world at
each of these virtual points. Then, the coordinates of these
points in real space relative to the real origin P ,Ri
as Px iy jz k .
is denoted
v =+ + We can compute the differTT
TT=+ += -vv
Ri Ri
,, ,,
Ri
Ri
ence in Cartesian coordinates between the holographic and
real points
() ,
Px iy jz kP P,,
Norm PP/^h (6)
i
=- Rivv 2
Hi
,, .
i ii iHiRi
along with the overall offset (the norm) and the average
overall offset across the entire sample:
Alignment
Offset Vector
∆Pi
Aligned
PR,i
Real Robot
Origin
PH,i
Holographic
Origin
Figure 8. An illustration of determination of the initial alignment
error, measured using a corresponding point in the holographic
IEEE Robotics & Automation Magazine - March 2022

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - March 2022
