IEEE Robotics & Automation Magazine - March 2023 - 19

GO TO LIVING ROOM
In the following scenario, the person sends requests using
voice assistance about his or her desire to go to the living
room. The global intelligence processes the person's statistics
(level of walking discomfort and history of similar requests)
and suggests that a walker robot will be suitable to use this
time. Then, the global intelligence sends the request to the
robot walker to complete the task. Based on the person's sitting
position on the bed, an appropriate location where the
walker robot should be sent is given as a goal. At the same
time, the global intelligence adjusts the automated bed height
such that the person can comfortably get off the bed with
maximum ease. It conveys to the user that the bed height is
adjusted and that the robot is on its way. The walker robot
stops at the desired location and activates its brakes to avoid
wheel slippage. Using camera information, the monitoring
system in the global intelligence can predict that the standing
task is completed by utilizing the information from the force
sensor plates installed under the bed.
Moreover, the global intelligence uses the force and IMU
sensors [28] and a safety model on the walker handrail grip to
confirm that the user is in the correct position and releases the
brake. Then, the robot goes to shared control mode to assist
and support the user to the intended place. In the next step,
once the user has released his or her grip, the robot senses
that the user has completed the task, and the computer sends
the robot back to its base station (as shown in Figure 10, time
stamp t = 2:53).
To demonstrate how our system can work along with a
human caretaker, another experiment is conducted as a continuation
of the scenario. This scene introduces a human caretaker
with control of the robots' operation. This experiment
aims to demonstrate how the system can take commands from
different users and distinguish between the care receiver and
caregiver. The global intelligence keeps a database of different
users and, based on where a request is coming from (AI-based
voice synthesis, microphone localization, and image recognition),
completes the request through the most appropriate
selection. The snapshot t = 3:44 in Figure 10 demonstrates
some sensory system information from the scene. Here, there
is an environment classifier; we utilize the YOLO v3 deep
learning network [27] to distinguish objects in the scene, and
the classifier gives the semantic information of the objects and
people. The system tracks the two people in the scene and can
pinpoint the request source.
For example, the caretaker wishes to send medicine to the
care receiver. In the scene, he or she calls the robot to his or
her location. The person's position is extracted by information
matching and gathering from where the request originated. The
global intelligence picks an appropriate robot for the task and
sends the robot to the caller's location. Next, the caretaker puts
the medicine on the service robot and asks the global intelligence
to send it to another person. The global intelligence
accepts the request, processes the voice command for key
information (e.g., the person's name), and utilizes the camera
network and stored database to recognize the person and his
or her position on the map. It then sends the same robot to the
other person and waits for the task to be completed. The process
happens instantaneously, and there is no delay in the communication
and message exchange with our proposed system.
GOING OUTSIDE
This scenario demonstrates how the CARE system can be
flexibly extended to cases outside the boundaries of a home,
t = 0:01
t = 0:20
(a)
(b)
t = 0:35
(c)
t = 0:45
t = 0:57
(d)
(e)
t = 1:15
(f)
FIGURE 9. The bedroom scene. (a) User wakes up. (b) The automated robot bed adjusts itself. (c) The user requests a drink from global
intelligence (computer) using voice assistance. (d) The bed height is adjusted for the user. (e) The service navigates autonomously in the
facility to execute the task. (f) User picks up the drink from the robot.
MARCH 2023 IEEE ROBOTICS & AUTOMATION MAGAZINE
19

IEEE Robotics & Automation Magazine - March 2023

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - March 2023