IEEE Robotics & Automation Magazine - September 2023 - 57

INTRODUCTION
Haptics improve teleoperation quality by enhancing the user
experience with tactile and force feedback [1], [2], [3], [4].
Haptic devices have been extensively used in many fields,
from medicine [5] to industry and service. Measuring the
forces acting on the teleoperated system is at the core of haptics.
When feasible, an onboard force sensor enables a direct
force estimation. However, many applications prevent this
approach because of, for example, space constraints (e.g.,
microrobots), hazardous environments (e.g., radioactive
areas), and the unreliability of conventional sensors (e.g., continuum/compliant
robots, where a force sensor on the tip
would measure not only external forces but also internal reaction
forces, due to the elasticity of the system, corrupting or
hiding the actual external load).
To address those cases, indirect force sensing is widely
researched to extract the load acting on the system through a
different kind of measurement. For example, fiber Bragg grating
(FBG) can sense force through optical features [6], [7],
[8]. However, FBG sensors must be embedded in the robot's
body [9] or end effector [10], [11]. This requires the robot to be
designed or modified to integrate the sensors, which could be
unfeasible and highly expensive. As an alternative, vision-based
force measuring methods use the deformation of the robot's
structure [12] or surroundings [13] caused by contact forces to
estimate the force exchange. However, this method requires at
least one of the contact surfaces to be soft enough to obtain visible
deformation and assumes an unhindered line of sight. Other
applications employ " remote " force measurements to estimate
the load acting on the end effector of a robot. This is done, for
example, in tendon-driven continuum robots, where load cells
at the tendons are used to estimate the force at the tip [12], [13],
[14]. This method is accurate only in specific conditions (i.e., an
external load comparable with the tendon tension), as the load
could be " hidden " by signal noise and other factors. Furthermore,
an accurate dynamic model of the robot is required, and
solving this model could prevent real-time applications.
Bilateral Control
System
More extreme cases, such as operation in radioactive environments
[15], [16] and in extremely confined spaces (e.g.,
airplane engines [17]), could prevent the use of onboard sensors
altogether. When these operations generate an audible
sound, however, this acoustic signal can be acquired with an
external remote system and processed to extract information
from the operation itself. For example, in machining [18],
the frequency signatures of audible sounds have been shown
to identify cutting [19], sawing [20], milling [21], and grinding
[18] parameters and evaluate surface quality [22].
These examples show that acoustic methods can be successful
in acquiring process information from audible features.
We thus propose, in this work, a haptic system that
provides feedback in scenarios where the work environment
cannot be accessed and seen but where a sound proportional
to process force/torque can be acquired remotely.
This method is independent from the system, as it does
not require any hardware modification. As illustrated in
Figure 1, a haptic interface works as a leader device to send
motion commands and control a follower robot. Meanwhile,
process sound is acquired and used to extract the
interaction force from its audible features. The system identifies
the corresponding force feedback and realizes it with
the haptic interface. An overview of the control method
is reported in the " System Overview " section. In the " An
Example: Haptic Control of Machining Tasks " section, an
example of the proposed system is reported for an aerospace
repair application, with the robot in Figure 1 guided
in a machining operation through a haptic device. In the
" Experimental Validation " section, an experimental validation
is presented for different machine tools and workpiece
materials, and the results are discussed.
The main contribution of this article is represented by
the development of a method that generates haptic feedback
for any sound-generating process in which the force is proportional
to the acoustic emission (e.g., machining). The
main challenge behind the proposed method is processing
Follower Robot
Motion
Command
Sound
Signal
Motion
Command
Haptic Interface
Force
Feedback
Confined
Access
No Visual
Feedback
Acquiring
Process
Sound
(a)
(b)
FIGURE 1. The process of obtaining haptic feedback from audible sound features. The (a) user/leader and (b) robot/follower.
SEPTEMBER 2023 IEEE ROBOTICS & AUTOMATION MAGAZINE
57
IEEE Robotics & Automation Magazine - September 2023

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2023
