IEEE Robotics & Automation Magazine - September 2023 - 58

TABLE 1. The nomenclature used in this article.
VARIABLE DESCRIPTION
System Overview
PI
RI
PE
RE
TIE
li
n
PI
RI
lS
1
S
S
F
M
PI
RI
AI
AI
WI
FI
MI
WE
FE
ME
Ci
W ,Ei
gS
x
~
f
Ws
AS
ii
L
tx
ty
tz
FH
I
PH
E
PH
I
kI
FM
REtoH
kII
PH
S
kIII
%r
R
fen
f
f
3
f
f
M
M
Input position of the haptic device (haptic frame)
Input orientation of haptic device (haptic frame)
Desired end-effector position (robot frame)
Desired end-effector orientation (robot frame)
Transformation from haptic to robot frame
Desired position of robot actuators
Number of degrees of freedom of the robot
Current position of the haptic device (haptic frame)
Current orientation of haptic device (haptic frame)
Current position of robot actuators
Current robot end-effector position (haptic frame)
Current robot end-effector orientation (haptic frame)
Force stiffness matrix for motion control
Moment stiffness matrix for motion control
Haptic feedback wrench from motion control
Haptic feedback force from motion control
Haptic feedback moment from motion control
Wrench applied to the robot end effector
Force applied to the robot end effector
Moment applied to the robot end effector
Measurable sound feature
Components of WE
Spindle torque
Spindle rotation velocity
Sound wave frequency
Haptic feedback wrench from touching the sound
Scaling factor for touching the sound wrench
Robot Teleoperation
Bending angle of the ith compliant joint of the robot
Actuation of the linear drive of the robot
Scaling factor from 1i to the x-axis of the haptic device
Scaling factor from 2i to the y-axis of the haptic device
Scaling factor from L to the z-axis of the haptic device
Haptic Feedback
Haptic feedback force for the example system
Haptic device position vector (haptic frame)
calculated from the current robot actuation
Commanded position (haptic frame)
Scaling factor for motion control mode
Radial machining force
Rotation matrix from the robot to the haptic frame
Scaling factor of machining force feedback
Haptic device position when entering an alert area
Stiffness of the haptic " virtual wall "
Haptic device motion direction
Force vector mapped in the haptic frame
Sound Tracking Algorithm
pi,
pi, 1Environment
noise frequency upper limit (1,000 Hz)
Frequency with the highest amplitude in the current loop
Frequency with the highest amplitude in the
previous loop
out,i
out,i 1Frequency
change limit in consecutive loops (250 Hz)
Output spindle frequency in the current loop
Output spindle frequency in the previous loop
related to Ci
Mapping function between sound feature and
robot wrench
the sound signal for real-time feedback: previous works
demonstrated the feasibility of using sound to estimate
force features [18] but could not provide the high frequency
needed for haptic feedback and depended on " ideal "
acquisition conditions (microphones nearby the process,
controlled background noise, and no noise or disturbance
from a tailored experimental setup). In this article, we solve
these challenges to enable haptic teleoperation with the
proposed method, significantly increasing the refresh rate
of the sound processing algorithm and improving stability
in the presence of noise. This method can be adapted to
monitor any process in which a desired characteristic can
be extracted from sound. Since a single microphone, which
can be placed remotely, is needed to acquire sound, the
method has a significant advantage in removing the need to
assemble onboard sensors, which might not fit and involve
significant costs.
SYSTEM OVERVIEW
The system proposed herein (see Table 1) aims at enabling
teleoperation with haptic feedback for a robot that cannot
use conventional force sensors either for system requirements
or environmental constraints. To this end, we propose
using audible process features to generate haptic
feedback. The user could then sense the robot through a
single remote microphone.
The system is composed by three main parts, as shown in
Figure 2: the robot control, haptic interface, and sound signal
processing. The robot control block (A) receives commands
from the haptic device and controls the robot's actuators
accordingly. The sound signal processing block (B) acquires
the sound generated by the target operation and extracts the
force in real time. The haptic interface block (C) acquires
the user's motion input while providing him or her with force
feedback in real time.
The system can switch between two feedback modes:
motion control mode (the switch in block C linked to 1), when
no process sound is detected, and touching-the-sound mode
(the switch linked to 2), when an acoustic emission within
the process's range is acquired. In case 1, the haptic interface
provides force feedback only to limit a sudden input motion
that cannot be followed by the robot because of speed limits,
collisions, and safety. Thus, the haptic interface in this mode
provides a force proportional to the difference between the
current position of the robot and the desired one commanded
by the haptic interface: the larger the difference between the
robot's current and desired positions, the larger the force needed
to move the robot farther. In case 2, the system is touching
the sound. In this mode, the acoustic emission is processed to
derive the interaction force between the robot's end effector
and the environment, which is then scaled and returned to the
user as feedback on the process.
ROBOT MOTION CONTROL
When the system is in motion control mode, the user in -
puts the motion command from the haptic interface by
58 IEEE ROBOTICS & AUTOMATION MAGAZINE SEPTEMBER 2023
IEEE Robotics & Automation Magazine - September 2023

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2023
