IEEE Robotics & Automation Magazine - December 2022 - 106

performance of nonlinear systems, and the control law is
directly given in the Lyapunov stability analysis process [20]-
[23]. In [20], BLF is employed to ensure that a single-input-
output nonlinear system stays within the output constraint. In
[21], the properties of BLF are used to constrain parameters
of the unknown function in a compact superset and solve the
complex problem of NN-approximation conditions. Inspired
by these ideas, to address the aforementioned drawbacks and
guarantee the success of visual servoing tasks, a BLF is considered
in combination with IBVS to restrain the image feature
points that remain within the FoV and guarantee stability of
the visual servoing system.
Another problem is the uncertainty of system dynamics
due to inaccurate modeling of the robotic system, which
affects the stability and control accuracy of the visual servoing
control system. Various solutions have been suggested for this
problem, and the SMC is a feasible method to improve
robustness of the control system [24]-[27]. In [25], a control
method is proposed that combines the SMC and the Takagi-
Sugeno fuzzy system model, which can effectively compensate
for the nonlinear disturbance and uncertainty of the
robotic system. Khurram et al. [26] designed a fast, integral
terminal SMC to handle system uncertainties of the robotic
manipulator. In this research, an SMC control law incorporated
with IBVS is advanced to dispose of the uncertainties of
7-DoF redundant robotic manipulators. Motivated by these
works, the contributions of this article are summarized in the
following three points:
1) A humanoid control algorithm is developed with IBVS
control by kinematically reflecting the human-like swivel
motion to make the human-robot collaboration capable
for most of the typical industrial serial manipulators.
2) The BLF is innovatively introduced to design the controller
of the visual servoing system. With the controller constraint,
all control variables of the visual servoing system
are confined within the bounds of the formed BLF while
satisfying FoV constraints.
3) A combined IBVS SMC law is proposed to address the
uncertainties of a 7-DoF redundant robot manipulator
while elevating the stability of the visual servoing system.
xc
P = [x y z]T
yc
U
V
OI
Oc
zc
Center of Camera
λ
Center of
Image
Image Plane
d
where (, )[ (, ),f,( ,)]T
T is the timeLz
sL zs Lz sdm m
sL(, ),zs Vdc
T
o =
=
11 1
= f
1
dm
T
is the vision system's image-interaction
matrix, and [, ,]zz zdd dm
Figure 2. The geometric model of a pinhole camera.
106 * IEEE ROBOTICS & AUTOMATION MAGAZINE * DECEMBER 2022
varying depth vector. The velocities of the
camera Vc
and robot end effector Ve
can be
p = [u v]T
Modeling and Preliminary
System Dynamics
The dynamic model of an n-link rigid robotic system is usually
described as () (, )( ),
and qRn
Mq qC qq qG q x++ =
po o
where ,q ,qo
p ! are the joint angle position, joint angle velocity,
and joint angle acceleration variables, respectively. x ! Rn
are the torque control input variables, ()!Mq Rnn is the
inertia matrix of the robot, (, )Cq qq Rn
Gq Rn
#
oo ! is the centripetal
and Coriolis force matrix, and ()! is the gravitational
matrix. The system dynamics can be described as follows:
()[( ,)
dt
d q
== -
qo =
GG.
qo
Mq Cq qq Gq
1
x-- ()]
oo
Define the end-effector velocity as [] , where
y e and e~ are the translation and rotation velocity, respecVee
e
= y~ T
tively. A mutual conversion between the velocity of the end
effector and the joint velocity can be realized by utilizing the
Jacobian matrix, and the conversion relationship is
VJ () ,qqeq
=
o where ()!Jq R #6 n
q
is the robot Jacobian
matrix, and qo is the joint velocity of the robot manipulator.
We now consider the visual servoing system with a pinhole
camera model mounted at the robot's end effector. A
schematic diagram of the projection transformation model of
the pinhole camera is shown in Figure 2. []
xy zcc c
O .c
im1 f=
,,
s == m66
i
v
u
o =
i
T represents
the axes of vision system frame C attached at the center of
camera
Define a fixed set of 3D points as [] ,Px yzii ii
T
=
image feature points [] ,su vii i
T
=
(/ ),iz
The relationship between the time variation of the image feature
points and the camera velocity Vc
sL (, ),zs Vii ii c where
y
x
R
Lz s(, ) =
ii i
T
S
S
S
S
m
-
zi
m
-
zi
[]
vw
cc
z
u
z
v
i
i
i
i
m + vi
2
m
m
uv
ii
-
2
m + ui
m
2
uv
--ui
m
ii
2
vi
V
X
W
W
W
W
is the image-interaction matrix, Vc
T is the velocity of the camera, and
=
vc and wc are the linear and angular velocities,
respectively.
Define [, ,]ss sRT
= f ! as feature
1
m
T Tm
2
point vectors of the whole vision system,
and the differential image feature coordinate
is obtained as
i
i
in the camera frame; the corresponding 2D
im1 f=
,,
are given as
i@@ where m is the focal length of the camera.
can be written as
IEEE Robotics & Automation Magazine - December 2022

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2022
