IEEE Robotics & Automation Magazine - September 2020 - 43

Table 2. The 3D-reconstruction errors.
3D-Reconstruction Error (Triangulation)

fm (mm)

Trajectory
Experiment 1

fv (mm)

fRMS (mm)

3.79

2.92

2.99

2.15

1.65

1.82

2.58

1.98

2.07

Trajectory-planner error (QP versus actual)
Experiment 1

2.49

1.25

1.55

2.56

1.06

1.45

2.42

1.85

1.9

Overall results (experiments 1 and 2)
Optimization routine (rad/s)

1.24

Tracking (mm) Static environment
(mean error)

0.5 ± 0.3

1.57

Dynamic environ- 2.3 ± 0.6
ment (mean error)
Errors indicate the accuracy with which objects are triangulated inside
the ARMM system workspace. The ground truth for reconstruction
is the end-effector position estimated using the low-level controller
of each manipulator (UR10/UR5). The 3D-reconstruction error
(triangulation) results represent the maximum (fm), mean (fv), and
RMS (fRMS) errors (in millimeters) between the position as estimated
from angular data provided by the robot's encoders and the position
as tracked by the eight cameras. Trajectory-planner errors resulting
from the OBS-avoidance algorithm are indicated in radians per second
for the optimization routine, followed by the overall tracking errors of
both experiments.

magnetic-actuation system. However, one of the limitations
introduced in this study relates to conducting experiments
in a realistic, clinically relevant scenario. In reality, the
amplitude and period of breathing motions vary with time
and among patients. Furthermore, preoperative patient-vasculature data would be more relevant when investigating
accurate 3D catheter-tip positioning in vivo. Since the focus
of the experiments was on the task-space control of the
robotic end effectors, we have not investigated the accurate
magnetic control of a catheter tip based on real-time subsurface imaging. However, this leaves room for future studies
conducted with the ARMM system.
A second limitation of optical tracking is that it is only
useful for monitoring surgical instruments outside the body.
Moreover, tracking the instruments requires a direct line of
sight with at least two cameras. To solve this, sensor-based
fusion techniques can be investigated that combine optical
tracking and US imaging to record flexible objects inside the
body. Rigid bodies can be constructed from active markers
instead of the passive reflective markers used in this study.
Such markers emit radio frequency signals and can be integrated with accelerometer-gyroscope sensors. This would
enable the sensing of moving objects' angular velocities,
which can be useful for providing additional coordinate data
in case of occlusions. A potential drawback could be related
to the flexibility and size of our system: a fixed tracking

system is an important requirement; hence, optimally positioning the cameras may be time consuming. Recalibration
of the tracking system can also be challenging since it consists of eight cameras as opposed to single stereo-camera
trackers, such as the NDI Polaris. Fortunately, the Motive
software can adjust the system calibration to account for
changes, even drastic ones, in the positions and orientations
of the cameras.
Improving the mobility of the ARMM operating table
may be challenging due to its size (Figure 2). However,
in practice, the entire system can be transported between
operating rooms since
each of the subsystems
The field is far from
is mobile. Such a scenario only implies that
maturity, and the benefits
an additional calibration
step would be necessary,
of surgical robots are still
as explained in the "Preoperative Planning: Worka subject of debate in
space Registration" section.
Finally, the magnetic field
medical communities.
generated by the EM coil
may affect the serial-link
manipulator's encoders.
Since the encoders used in the joints of the UR10 are
magnetic, they can be damaged when high external magnetic fields are present. We took this into account when
we designed the electromagnet and calculated its offset
from the end-effector surface. The electromagnet has
been tested at a maximum of 15 A, indicating no malfunction. As a safety measure, in this study's experiments, we
did not exceed ±6 A.
Conclusions and Future Work
We presented a tracking and navigation technique that aims
to bring the ARMM system a step closer to clinical studies.
By incorporating two surgical robots, the system reduces the
constraints of similar actuation systems, including an insufficient workspace, limited DoFs for surgical tools, and bulky
components. Furthermore, we introduced an obstacleavoidance strategy based on marker-aided optical tracking.
We demonstrated the 3D reconstruction accuracy of the
tracking system and the velocity errors of the optimization
routine. The navigation to target robot poses was accomplished around static and dynamic obstacles with an overall 3D tracking accuracy of 0.5 ± 0.3 mm (static) and 2.3 ±
0.6 mm (dynamic). For the proposed optimization routine,
an error of 1.57 rad/s (RMSE) and a mean error of 1.24
rad/s were achieved.
We plan to demonstrate the clinical feasibility of the
ARMM system by remotely steering a magnetically actuated
catheter inside a realistic US phantom. Such a phantom
should represent real arteries, including simulated blood
flow and breathing motions. Improved magnetic-catheter
designs can be tested using the ARMM system, and the
availability of more than two manipulators may enable
SEPTEMBER 2020

IEEE ROBOTICS & AUTOMATION MAGAZINE

IEEE Robotics & Automation Magazine - September 2020

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2020