IEEE Robotics & Automation Magazine - December 2011 - 39

The planar controller and state estimator operate at a low
level. The control and estimate is calculated for each pair of
images from the cameras that are received at 7:5 Hz. Realtime tracking of the needle provides the (x, y, z) positions of
the needle tip. We calibrate the stereo cameras using the
camera calibration toolbox for MATLAB and assign a world
reference frame [22]. We compensate for the refraction of
the gel by computing the refractive index of the gel and use it
in the triangulation process. Since the tip is triangulated from
two cameras with limited resolution and noise, the height (z)
measurements are noisy. The model-based observer estimates the roll, pitch, and distance of the needle tip to the
desired plane over time. Of course, an encoder can read the
roll angle at the base of the needle, but that is not necessarily
the same as the angle at the tip even, with perfect alignment
of the actuator and the tip before insertion [5]. After triangulating the camera images, the controller adjusts the angle of
the needle base to guide the needle to the desired plane. The
controller is unconcerned about the position and yaw
(x, y, /) within the desired 2-D plane; it is the job of the
high-level motion planner. Likewise, the motion planner is
unconcerned with the height, roll, and pitch because it
assumes that the planar controller maintains them correctly.
At the start of each insertion, the user is asked to click
near the tip of the needle to provide a local region in which
a corner detector locates the needle tip. The needle is then
tracked during the insertion using the Brute Force tracker in
the computer-integrated surgical systems and technology
software libraries [23]. Throughout the motion, the system
monitors the triangulated position. If the deviation is too
large or the needle is too far away from the plane, the system
will pause the insertion and ask the user to reselect the location of the needle. A stop button will similarly pause the
insertion to allow the operator to verify the needle location
or abort the procedure by retracting the needle. This is a
safety feature to ensure that the needle tracking is not lost.
For the experiments described in the "Experimental Evaluation" section, we used stereo cameras, but clinical procedures (such as that discussed in the "Brachytherapy
Procedure" section) will use biplane fluoroscopy or threedimensional (3-D) ultrasound to directly measure the needletip position inside the tissue. Since the needle is too thin for
direct measurement of the tip orientation, we use the observer's estimate of orientation. The planar controller requires two
cameras for triangulation of the needle height; thus, a single
camera frame is insufficient to adequately maintain the planar
motion. When using a single C-arm fluoroscope, we anticipate
that an estimate of the 3-D position can be obtained based on
the information from two asynchronous camera sources, such
as two positions of the C-arm. Clinical systems could potentially use an estimation scheme based on alternating camera
positions to estimate the 3-D needle position.
Torsion Compensation During 180° Bevel Flips
The motion planner and planar controller, previously
described, are based on the kinematic model of needle

steering [3], which does not account for the lag between
the base angle and tip angle resulting from torsional friction at the needle-tissue interface and the torsional compliance of the needle. This effect is substantial during the
180 rotations (bevel flips) commanded by the motion
planner. The lag has been shown to be as high as 45 for a
needle inserted 10 cm into an artificial tissue similar to the
one used in our experiments [5]. Even a small discrepancy
is likely to result in poor performance or failure of the
motion planner and image-guided controller.
For the experiments performed here, we used a torsion
compensator that estimates and controls the needle tip
using a mechanics-based model of the rotational dynamics
of the needle interacting with the tissue during insertion
[5]. The dynamics are formulated with friction using a
fourth-order continuous forced modal model. The resulting control action of the compensator axially rotates the
needle at its base so that the tip angle is at the desired location. Without torsion compensation, the lagged rotation of
the needle tip would result in the needle deviating significantly out of the desired plane of motion.
The torsion compensator is updated at 7:5 Hz but operates at 200 Hz to continuously maintain the tip at the desired
angle and, thus, help maintain the needle in the desired plane.
The 180 bevel flips commanded by the motion planner and
the small adjustments commanded from the planar controller
are implemented using the torsion compensator.
Experimental Evaluation
Our needle-steering system is composed of the components
described in the "Components for Needle Control" section,
each of which has been individually tuned to achieve good
performance in the previous work. However, experimental
evaluation of a complete system that integrates planning, realtime control, and torsion compensation has never previously
been reported in the archival literature. An initial demonstration of our integrated system with discrete motion intervals
was presented at the IEEE International Conference on Biomedical Robotics and Biomechatronics in 2008 [1]. Here, we
describe experiments of continuous insertion designed to
determine the influence of each component's contribution in
driving the needle to the desired target.
Several assumptions were made during the implementation of the individual system components. Our goal in the
following experiments was to understand the extent to which
these assumptions are valid and what trade-offs the assumptions introduce. In particular, we will explore the system
performance when specific assumptions made for each
component are violated.
The experiments presented here use artificial tissues to
allow consistency from trial to trial; a biological tissue has
structures, such as muscle fibers and blood vessels, that can
alter the needle motion, making it difficult to ascertain the
extent to which our underlying systems-level assumptions
limit performance. There is still some variability in the needle path, but recent closed-form models validated with
DECEMBER 2011

IEEE ROBOTICS & AUTOMATION MAGAZINE

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2011