IEEE Robotics & Automation Magazine - December 2016 - 119

Automatic Ultrasound Acquisition
Medical ultrasound is an important imaging modality that
can provide real-time evaluation of patients. Compared with
many other modalities, an ultrasound scanner is portable
and substantially lower in cost, and it does not use harmful
ionizing radiation. In many intraoperative cases, ultrasound
is especially useful, as it can guide the surgery by providing
real-time images of otherwise hidden devices and anatomy.
In this study, we were particularly interested in TEE, which
can be used for diagnosing heart disease and guiding cardiac surgical procedures using either two-dimensional (2-D)
or three-dimensional (3-D) ultrasound imaging [1], [2].
However, similar to acquisitions of extracorporeal ultrasound, the operation of this technique still requires manual
control, which greatly depends on the onsite operator. The
onsite operation of ultrasound systems usually exposes
operators to an undesirable working environment that could
potentially cause occupational injuries [3]. Particularly for
TEE in minimally invasive interventional procedures, the
operator may be exposed to ionizing radiation from X-ray
C-arms and have to wear uncomfortable and bulky shielding [4]. These problems may be addressed by remotely operated ultrasound systems employing robotic techniques. Our
previous design of the first TEE robot demonstrated the
capability of remote control [5].
However, remote-controlled ultrasound systems are not
adequate to solve the problem of general usability of medical
ultrasound, as those systems still require a manual control
approach that is tedious, time consuming, and, most of all,
entirely operator dependent. In both developed and developing countries, because of the requirement for highly specialized skills during ultrasound acquisition, small clinics in
remote locations are unlikely to have ultrasound experts available most of the time [6]. It has been suggested that the lack of
ultrasound training is one of the biggest barriers to reliable
image acquisitions [7], [8]. Skill and training issues are made
worse since the views of the heart can be relatively complicated to understand. Therefore, intelligence toward automatic or
semiautomatic ultrasound acquisitions has to be built into
these robotic medical ultrasound systems to achieve a breakthrough in the utilization of ultrasound with robots.
There are a few existing publications on robotic-assisted
acquisitions for extracorporeal ultrasound [9]. Several works
have explored visual servoing techniques for tracking particular features, for example, tracking the carotid artery using
an extracorporeal ultrasound robot [10]. Additional projects
have been extended by other researchers to automatic scanning of the carotid artery using motion compensation [11].
The idea of a vision-based, self-guided system is also
employed in other types of medical robots, such as an eye-inhand endoscopic robotic system that is used for physiology
motion rejection in natural orifice transluminal endoscopic
surgery, where the robot is used for tracking an area of interest despite breathing motion [12]. In other robotic domains,
visual servoing is also one of the most important techniques
widely used for manufacturing, object picking, teleoperation,

missile tracking, and so forth. A comprehensive review of
those works is given in [13]. However, using visual servoing
techniques based on feature extraction for cardiac ultrasound acquisitions is challenging due to the complexity of
the heart structure. Because global information of the heart
from magnetic resonance (MR) or computed tomography
(CT) images always exists in many cardiac surgical procedures, we believe that using preplanned motion and multimodality registration is the preferred approach to deal with
vision and motion. To our knowledge, no such system has
been proposed for cardiac echocardiography, and we present
a unique study providing a complete solution for automatic
imaging of the heart.
In this study, we provided a view-planning solution for
transesophageal ultrasound views in patient-specific data
during cardiac procedures based on a preprocedure CT or
MR imaging (MRI) scan. We also developed a method for
automatic acquisition of these planned views using our
robotic system. This method, though primarily designed for
TEE, has the potential to be adapted for other intraoperative
ultrasound, such as transrectal or intracardiac imaging.
Other aspects of the proposed TEE robot, including safety
and force sensing, were briefly discussed in our previous
paper [5]. Because this article focuses on automation and
tracking, the topics of safety and force sensing are not
included here.
Robotic Transesophageal Ultrasound System
We have previously developed an add-on robotic transesophageal ultrasound system [Figure 1(a)] to operate a
commercial TEE probe (X7-2t, Philips Healthcare, The
Netherlands), allowing operators to precisely adjust the
probe's position remotely. The slave system (the robot) is
driven wirelessly via Bluetooth communication and has
several mechanisms for holding and manipulating the original probe. Our previous work [5] on evaluating the precision of the robotic system in free space demonstrated a
high repeatability of the probe positioning with a mean
error of 0.6 mm and 0.3° for the translation axis, 1.3 mm
and 0.9° for axial rotation, and 0.6 mm and 0.8° for bidirectional bending. Since each individual degree of freedom
was driven by a different motor, multiple axes of the robot
could be driven simultaneously in any combination. We
have since developed a dummy probe that serves as the
master control device for intuitive remote operation of the
robot [Figure 1(b)]. This dummy probe has a similar shape
to the original TEE probe handle and has built-in sensors
to interpret users' inputs. A pair of buttons on the dummy
probe is used for driving the linear belt mechanism of the
robot for translating the original probe. A rotating handle,
which can rotate about the long axis of the dummy probe,
is used for driving the gear train mechanism of the robot
for rotation of the original probe. A dual-shaft driving knob
pair is used for rotating similar knobs on the original probe
handle, via the belt mechanism, for bidirectional bending
of the probe head.
DECEMBER 2016

IEEE ROBOTICS & AUTOMATION MAGAZINE

119

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2016