IEEE Robotics & Automation Magazine - March 2023 - 15

to reach one another by using webhooks formed with one
another's Internet Protocol addresses. To simplify naming
and enable the robots to communicate from any network, we
use ngrok, which is a cross-platform application that enables
developers to securely expose a local webserver to the Internet
with minimal effort. This requires only that a robot has a
connection to the Internet. In this way, robots can address
one another using webhooks formed with the URL assigned
via ngrok.
INTERACTION BETWEEN SYSTEMS AND IoT
As explained before, to trigger a service in a robot, a web
request with a JSON payload is used to trigger a webhook,
and the robot will process the request-and-call service in its
own ROS core. This enables us to trigger these services from
another robot and any device that can generate a web
request, such as voice assistants (Siri and Alexa), Internet
services (IFTTT), and even a web browser. Because the
interface enables the robot to trigger remote webhooks (outgoing
information), it can also trigger IFTTT webhooks so
that the robot can interact with any of the 700+ web services
provided via IFTTT.
HUMAN INTERACTION WITH AI
The CARE framework, including the different subsystems,
is presented in Figure 5. There are three main systems:
global intelligence, local intelligence, and external sensors
with human interface systems. The global intelligence is
responsible for collecting user commands and processing
them to detect and track participants in the environment by
using a previous database of individuals. The next step is to
determine the available and suitable robots for tasks.
Autonomous robots perform tasks based on their structure,
sensors, and motors within the local intelligence. Each of
the components of the framework is discussed in detail in
the following.
REQUEST UNDERSTANDING
Human and robot/guidance system interaction is one of the
key points in achieving successful executions based on
patient/user requests. However, these interactions have certain
bottlenecks since requests from a user might happen multiple
times, and the user/patient could change his or her opinion
during an ongoing task. Moreover, the safety and ergonomics
of the human interface are important; hence, the person can
interfere with the execution at any time he or she wants to.
The interactions are divided into direct and indirect parts
in this work. These interaction interfaces require different sensors
that help the general intelligence of the care system to act
responsively and on time. For direct interaction, a physical user
input device, such as a tablet/smartphone, and verbal interaction
through a commercially available virtual assistant, such as
Amazon Alexa voice services and Google voice assistance, is
utilized. When physically interacting with smartphone and tablet-based
communication, the user can send direct commands
to the available robots by using custom shortcut buttons. These
shortcuts consist of tasks with varying complexities. For example,
there is a complete task request for bringing drinks and for
sending a robot back to the base station. Another example is
bed assistance, where the user can adjust the bed to his or her
desired head angle and height. For voice-based communication,
apart from simple task requests, the person can interact
with a virtual assistant through normal conversational sentences
to ask for service and assistance from various robots.
For indirect interaction, different sensory systems are
placed in the environment. These include fixed cameras for
person tracking and recognition, a motion capture system
for precision tracking and analyzing of body posture, force
plate sensors mounted near the bed and sofa for calculating
standing/sitting force, and a RGBD camera system mounted
over the bed for pose estimation and performing sleep analysis
patterns. Please note that we maintain a database of the
Living Lab users, whose information (age, gender, face, voice
IFTTT
Siri
RobotA
Internet
Publish to
Topic
ROS Core
Actions/
Services
Webserver
WebBrowser
FIGURE 4. The structure of the developed communication interface.
MARCH 2023 IEEE ROBOTICS & AUTOMATION MAGAZINE
15
Webserver
Actions/
Services
Publish to
Topic
RobotB
ROS Core
ngrok
ngrok
IEEE Robotics & Automation Magazine - March 2023

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - March 2023
