Tech Briefs Magazine - June 2023 - MD-19

Teaching Robots How to
Predict Human Preferences in
Assembly Tasks
This technology could also be useful in industrial settings where
workers are tasked with assembling products on a mass scale,
saving time, and reducing the risk of injury or accidents.
University of Southern California, CA
H
umans have a way of understanding
others' goals, desires and beliefs, a
crucial skill that allows us to anticipate
people's actions. Taking bread out of the
toaster? You'll need a plate. Sweeping up
leaves? I'll grab the green trash can. This
skill, often referred to as " theory of mind, "
comes easily to us as humans, but is still
challenging for robots. But, if robots are
to become truly collaborative helpers in
manufacturing and in everyday life, they
need to learn the same abilities.
In a new paper, a best paper award finalist
at the ACM/IEEE International
Conference on Human-Robot Interaction
(HRI), USC Viterbi computer science researchers
aim to teach robots how to predict
human preferences in assembly tasks,
so they can one day help out on everything
from building a satellite to setting a table.
" When working with people, a robot
needs to constantly guess what the person
will do next, " said lead author Heramb
Nemlekar, a USC computer science
PhD student working under the supervision
of Stefanos Nikolaidis, an Assistant
Professor of computer science. " For example,
if the robot thinks the person will
need a screwdriver to assemble the next
part, it can get the screwdriver ahead of
time so that the person does not have to
wait. This way the robot can help people
finish the assembly much faster. "
But, as anyone who has co-built furniture
with a partner can attest, predicting
what a person will do next is difficult:
different people prefer to build the
same product in different ways. While
some people want to start with the most
difficult parts to get them over with, others
may want to start with the easiest
parts to save energy.
Most of the current techniques require
people to show the robot how they would
like to perform the assembly, but this takes
time and effort and can defeat the purpose,
said Nemlekar. " Imagine having to
Motion Design, June 2023
assemble an entire airplane just to teach
the robot your preferences, " he said.
In this new study, however, the researchers
found similarities in how an
individual will assemble different products.
For instance, if you start with the
hardest part when building an Ikea sofa,
you are likely to use the same tact when
putting together a baby's crib.
So, instead of " showing " the robot their
preferences in a complex task, they created
a small assembly task (called a " canonical "
task) that people can easily and quickly
perform. In this case, putting together
parts of a simple model airplane, such as
the wings, tail, and propeller.
The robot " watched " the human complete
the task using a camera placed directly
above the assembly area, looking
down. To detect the parts operated by the
human, the system used AprilTags, similar
to QR codes, attached to the parts.
Then, the system used machine learning
to learn a person's preference based
on their sequence of actions in the canonical
task.
" Based on how a person performs the
small assembly, the robot predicts what
that person will do in the larger assembly, "
said Nemlekar. " For example, if the robot
sees that a person likes to start the small
assembly with the easiest part, it will predict
that they will start with the easiest part in
the large assembly as well. "
In the researchers' user study, their
system was able to predict the actions
that humans will take with around 82
percent accuracy.
" We hope that our research can make it
easier for people to show robots what they
prefer, " said Nemlekar. " By helping each
person in their preferred way, robots can
reduce their work, save time and even
build trust with them. "
For instance, imagine you're assembling
a piece of furniture at home,
but you're not particularly handy and
19
http://info.hotims.com/84480-745
Tech Briefs Magazine - June 2023

Table of Contents for the Digital Edition of Tech Briefs Magazine - June 2023
