Tech Briefs Magazine - June 2024 - MD21

Deep Learning Models Enable
Robotic Motion
A method connects robot motion data with the " common sense
knowledge " of large language models.
Massachusetts Institute of Technology, Cambridge, MA
M
IT engineers are aiming to give robots
a bit of common sense when
faced with situations that push them off
their trained path. They've developed a
method that connects robot motion data
with the " common sense knowledge " of
large language models, or LLMs.
Their approach enables a robot to logically
parse many given household tasks into
subtasks, and to physically adjust to disruptions
within a subtask so that the
robot can move on without having to go
back and start a task from scratch - and
without engineers having to explicitly
program fixes for every possible failure
along the way.
" Imitation learning is a mainstream
approach enabling household robots.
But if a robot is blindly mimicking a human's
motion trajectories, tiny errors
can accumulate and eventually derail
the rest of the execution, " said Yanwei
Wang. " With our method, a robot can
self-correct execution errors and improve
overall task success. "
The researchers illustrate their new approach
with a simple chore: scooping marbles
from one bowl and pouring them into
another. To accomplish this task, engineers
would typically move a robot through the
motions of scooping and pouring - all in
one fluid trajectory. They might do this
multiple times, to give the robot a number
of human demonstrations to mimic.
The team realized that, while a human
might demonstrate a single task in one
go, that task depends on a sequence of
subtasks, or trajectories.
" That level of planning is very tedious, "
Wang said.
Instead, he and his colleagues found
some of this work could be done automatically
by LLMs. These deep learning
models process
immense libraries of
text, which they use to establish connections
between words, sentences, and
paragraphs. Through these connections,
an LLM can then generate new sentences
based on what it has learned
about the kind of word that is likely to
follow the last.
" LLMs have a way to tell you how to do
each step of a task, in natural language.
A human's continuous demonstration is
In this collaged image, a robotic hand tries to scoop up red marbles and put them into another bowl while a
researcher's hand frequently disrupts it. The robot eventually succeeds. (Image: Jose-Luis Olivares, MIT. Stills
courtesy of the researchers)
Motion Design, June 2024
21

http://info.hotims.com/86256-729

Tech Briefs Magazine - June 2024

Table of Contents for the Digital Edition of Tech Briefs Magazine - June 2024

Tech Briefs Magazine - June 2024 - Intro
Tech Briefs Magazine - June 2024 - Sponsor
Tech Briefs Magazine - June 2024 - Band1
Tech Briefs Magazine - June 2024 - Band2
Tech Briefs Magazine - June 2024 - Cover1
Tech Briefs Magazine - June 2024 - Cover2
Tech Briefs Magazine - June 2024 - 1
Tech Briefs Magazine - June 2024 - 2
Tech Briefs Magazine - June 2024 - 3
Tech Briefs Magazine - June 2024 - 4
Tech Briefs Magazine - June 2024 - 5
Tech Briefs Magazine - June 2024 - 6
Tech Briefs Magazine - June 2024 - 7
Tech Briefs Magazine - June 2024 - 8
Tech Briefs Magazine - June 2024 - 9
Tech Briefs Magazine - June 2024 - 10
Tech Briefs Magazine - June 2024 - 11
Tech Briefs Magazine - June 2024 - 12
Tech Briefs Magazine - June 2024 - 13
Tech Briefs Magazine - June 2024 - 14
Tech Briefs Magazine - June 2024 - 15
Tech Briefs Magazine - June 2024 - 16
Tech Briefs Magazine - June 2024 - 17
Tech Briefs Magazine - June 2024 - 18
Tech Briefs Magazine - June 2024 - 19
Tech Briefs Magazine - June 2024 - 20
Tech Briefs Magazine - June 2024 - 21
Tech Briefs Magazine - June 2024 - 22
Tech Briefs Magazine - June 2024 - 23
Tech Briefs Magazine - June 2024 - 24
Tech Briefs Magazine - June 2024 - 25
Tech Briefs Magazine - June 2024 - 26
Tech Briefs Magazine - June 2024 - 27
Tech Briefs Magazine - June 2024 - 28
Tech Briefs Magazine - June 2024 - 29
Tech Briefs Magazine - June 2024 - 30
Tech Briefs Magazine - June 2024 - 31
Tech Briefs Magazine - June 2024 - 32
Tech Briefs Magazine - June 2024 - 33
Tech Briefs Magazine - June 2024 - 34
Tech Briefs Magazine - June 2024 - 35
Tech Briefs Magazine - June 2024 - 36
Tech Briefs Magazine - June 2024 - 37
Tech Briefs Magazine - June 2024 - 38
Tech Briefs Magazine - June 2024 - 39
Tech Briefs Magazine - June 2024 - 40
Tech Briefs Magazine - June 2024 - 41
Tech Briefs Magazine - June 2024 - 42
Tech Briefs Magazine - June 2024 - 43
Tech Briefs Magazine - June 2024 - 44
Tech Briefs Magazine - June 2024 - 45
Tech Briefs Magazine - June 2024 - 46
Tech Briefs Magazine - June 2024 - 47
Tech Briefs Magazine - June 2024 - 48
Tech Briefs Magazine - June 2024 - Cover3
Tech Briefs Magazine - June 2024 - Cover4
Tech Briefs Magazine - June 2024 - MDCover1
Tech Briefs Magazine - June 2024 - MDCover2
Tech Briefs Magazine - June 2024 - MD1
Tech Briefs Magazine - June 2024 - MD2
Tech Briefs Magazine - June 2024 - MD3
Tech Briefs Magazine - June 2024 - MD4
Tech Briefs Magazine - June 2024 - MD5
Tech Briefs Magazine - June 2024 - MD6
Tech Briefs Magazine - June 2024 - MD7
Tech Briefs Magazine - June 2024 - MD8
Tech Briefs Magazine - June 2024 - MD8A
Tech Briefs Magazine - June 2024 - MD8B
Tech Briefs Magazine - June 2024 - MD8C
Tech Briefs Magazine - June 2024 - MD8D
Tech Briefs Magazine - June 2024 - MD9
Tech Briefs Magazine - June 2024 - MD10
Tech Briefs Magazine - June 2024 - MD11
Tech Briefs Magazine - June 2024 - MD12
Tech Briefs Magazine - June 2024 - MD13
Tech Briefs Magazine - June 2024 - MD14
Tech Briefs Magazine - June 2024 - MD15
Tech Briefs Magazine - June 2024 - MD16
Tech Briefs Magazine - June 2024 - MD17
Tech Briefs Magazine - June 2024 - MD18
Tech Briefs Magazine - June 2024 - MD19
Tech Briefs Magazine - June 2024 - MD20
Tech Briefs Magazine - June 2024 - MD21
Tech Briefs Magazine - June 2024 - MD22
Tech Briefs Magazine - June 2024 - MD23
Tech Briefs Magazine - June 2024 - MD24
Tech Briefs Magazine - June 2024 - MDCover3
Tech Briefs Magazine - June 2024 - MDCover4
https://www.nxtbook.com/smg/techbriefs/24TB12
https://www.nxtbook.com/smg/techbriefs/24TB11
https://www.nxtbook.com/smg/techbriefs/24TB10
https://www.nxtbook.com/smg/techbriefs/24TB09B
https://www.nxtbook.com/smg/techbriefs/24TB09
https://www.nxtbook.com/smg/techbriefs/24TB08
https://www.nxtbook.com/smg/techbriefs/24TB07
https://www.nxtbook.com/smg/techbriefs/24TB06
https://www.nxtbook.com/smg/techbriefs/24TB05B
https://www.nxtbook.com/smg/techbriefs/24TB05
https://www.nxtbook.com/smg/techbriefs/24TB04
https://www.nxtbook.com/smg/techbriefs/24TB03
https://www.nxtbook.com/smg/techbriefs/24TB02
https://www.nxtbook.com/smg/techbriefs/24TB01
https://www.nxtbook.com/smg/techbriefs/23TB12
https://www.nxtbook.com/smg/techbriefs/23TB11
https://www.nxtbook.com/smg/techbriefs/23TB10
https://www.nxtbook.com/smg/techbriefs/23TB09
https://www.nxtbook.com/smg/techbriefs/23TB08
https://www.nxtbook.com/smg/techbriefs/23TB07
https://www.nxtbook.com/smg/techbriefs/23TB06
https://www.nxtbook.com/smg/techbriefs/23TB05
https://www.nxtbook.com/smg/techbriefs/23TB04
https://www.nxtbook.com/smg/techbriefs/23TB03
https://www.nxtbook.com/smg/techbriefs/23TB02
https://www.nxtbook.com/smg/techbriefs/23TB01
https://www.nxtbook.com/smg/Testing/22TB12
https://www.nxtbook.com/smg/techbriefs/22TB12
https://www.nxtbook.com/smg/techbriefs/22TB11
https://www.nxtbook.com/smg/techbriefs/22TB10
https://www.nxtbook.com/smg/techbriefs/22TB09
https://www.nxtbook.com/smg/techbriefs/22TB08
https://www.nxtbook.com/smg/techbriefs/22TB07
https://www.nxtbook.com/smg/techbriefs/22TB06
https://www.nxtbook.com/smg/techbriefs/22TB05-P
https://www.nxtbook.com/smg/techbriefs/22TB05-D
https://www.nxtbook.com/smg/techbriefs/22TB04
https://www.nxtbook.com/smg/techbriefs/22TB03
https://www.nxtbook.com/smg/techbriefs/22TB02
https://www.nxtbook.com/smg/techbriefs/22TB01
https://www.nxtbook.com/smg/techbriefs/21TB12
https://www.nxtbook.com/smg/techbriefs/21TB11
https://www.nxtbook.com/smg/techbriefs/21TB10
https://www.nxtbook.com/smg/techbriefs/21TB09
https://www.nxtbook.com/smg/techbriefs/21TB08
https://www.nxtbook.com/smg/techbriefs/21TB07
https://www.nxtbook.com/smg/techbriefs/21TB06
https://www.nxtbook.com/smg/techbriefs/21TB05
https://www.nxtbook.com/smg/techbriefs/21TB04
https://www.nxtbook.com/smg/techbriefs/21BT03
https://www.nxtbook.com/smg/techbriefs/21TB02
https://www.nxtbookmedia.com