IEEE Robotics & Automation Magazine - December 2021 - 64

There are different approaches that can be used to automate
surgical tasks; for example, involving a human in a preplanning
stage, utilizing control theory to follow a human
during the operation, and using unsupervised reinforcement
learning or supervised machine learning to learn behaviors or
motions from human-provided examples and later executing
them autonomously. We decided to group efforts in RMIS
automation based on the application of the proposed control
strategy into the following control categories: general, instrument,
and camera.
Automation
68 Publications
Training, Skill
Assessment, and
Gesture Recognition
33 Publications
Hardware
Implementation
and Integration
99 Publications
Research Field Overview
Implementation of
Algorithms and Control
Strategies Toward the
Automation of Surgical
Subtasks to Decrease
the Surgeon's
Workload
General Control
Instrument Control
Camera Control
Study Gestures,
Assess Technical Skills,
and Augment Leaning
to Improve
Surgical Training
Modifications to the
dVRK Hardware
to Improve
Surgical Performances
Intraoperatively
Applications
Training Platforms
and Protocols
Training Augmentation
Skill Assessment
Workflow Analysis
dVRK Platform
Implementation
Haptics/
Pseudohaptics
New Surgical Tools
New Control Interfaces
Surgical Workflow
New Applicative
Domains
Outcomes
Automation is introduced
into surgical practice
by using data-driven
path planning,
modeling and algorithms
learning from expert's
demonstration. Shared
control between human
and robot as well as
the standardization of
safety measures can
be the first steps.
Big data and machine
learning are exploited
toward objective
mentoring (i.e., technical
skill assessment, and
feedback) in
simulation-based training
(involving both virtual
simulators and highfidelity
physical
mannequins) to take
the patient out of the
learning process.
The implementation
benefits from an open
platform like the dVRK
to integrate novel
technologies (that can
furtherly reduce
invasiveness) and
improve hardware to
meet usability needs
(e.g., improved HRI)
and cover gaps in
the current surgical
robotics platforms
(e.g., the lack of haptic
feedback).
Simulation is key in
moving toward
the new generation
or robotics platforms
without harming any
patient during testing.
Reaching a realistic
interaction between soft
objects and the robot is
the challenge, but
modeling and
advancements in
computer graphics can
boost the realism.
Vision is currently the
primary sensorial
information during da
Vinci-like surgery.
Great effort is invested
into processing and
augmenting images by
deep learning methods
to extract further
information and
enhance the surgeon's
awareness and
performance.
Spatial Mapping and
Understanding
Novel Imaging
Capabilities
Parameterization
Realistic
Robot-Object
Interactions
Camera and
Hand-Eye Calibration
Instrument and
Tissue Detection
and Segmentation
Integration of the dVRK
Into Simulation
Environments
(Without any Specific
Application
in the Publication Itself)
Process Images From
the dVRK Camera to
Extract Information and
Improve the Surgeon's
Performance
Intraoperatively
System Simulation
and Modeling
Seven Publications
Imaging and Vision
32 Publications
Figure 5. (From top to bottom) Each research field addressed is broken down into a general-concept overview, subdivision of
approaches or methodologies, and key potential outcomes or trends. HRI: human-robot interaction.
64 * IEEE ROBOTICS & AUTOMATION MAGAZINE * DECEMBER 2021
IEEE Robotics & Automation Magazine - December 2021

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2021
