IEEE Robotics & Automation Magazine - March 2022 - 27

Root
(475, 178, 169, 149)
Locality
CDN = " locality:navigation "
(440, 687, 631, 522)
Pick/Place
(1,721, 65, 121, 256)
Open/Close
Turn On/Off
Slice
(53, 0, 0, 0)
Slice
(360, 389, 351, 209)
Turn On/Off
Slice
(262, 28, 26, 84) (804, 0, 0, 0) (0, 0, 0, 0)
Slice
(0, 0, 0, 0)
Navigation
(0, 0, 0, 0)
Open/Close
Turn On/Off
(71, 0, 0, 0) (0, 0, 0, 0)
Slice
(0, 0, 0, 0)
CDN = " locality:navigation:pickplace:openclose:turnonoff:slice "
Figure 4. The search tree for a household environment. Each box represents a node N. The Nid
Examples of the CDN
Numbers convey how many Pid
W when " locality " in each CDN
component of each node is in bold.
components are shown beside nodes. Locality represents the set {kitchen, living room, bedroom, bathroom}.
are included in N
is changed for one of the values in the set. Robot
capabilities, such as navigation, picking and placing, and others, are used from the second hierarchy level onward.
vector representations, such as skip-grams and the Continuous
Bag of Words, have proved to be better abstractions that capture
relational meaning and synonymy that other word representations
cannot [34]. Such word representations are fast and
easy to train. However, it is also possible to use precomputed
word embedding, including the Global Vectors model [35].
We harness NIP
to train our word embedding model.
RNNs are a popular for processing variable-length input
Unlike traditional feed-forward networks, RNNs can
(8).
The interactions among the modules mentioned in this section
are illustrated in Figure 5.
Ranking
Given the context of the user (locality and robot capabilities),
in the previous stage, we retrieved a group of plans that
involved a similar context. Now we are interested in ordering
this group of plans based on how semantically similar the
schemes are when compared against the current configuration
of the user's environment. For this, we chose to create a
vector representation (sentence embedding) for each of the
plans referenced by ,W computed from the objects the
schemes require to be completed N(
identified in the environment O(
I )P
and for the objects
S). We use the weighted
average of the word vectors provided by our word embedding
MARCH 2022 * IEEE ROBOTICS & AUTOMATION MAGAZINE *
27
2
3
4
5
6
7
8
10
11
12
13
14
15
16
17
data, such as the sequence of word embedding derived from
O .S
represent variable-length input sequences by using a fixed set
of parameters. Long short-term memory (LSTM) is an RNN
architecture that has been successful in practice. We used this
architecture to implement our sequence model. The output of
LSTM is the input to the dense layer, which outputs the probabilities
per locality class label in L. (AP)2 selects the locality
class label with the highest probability and assigns it to LS
Algorithm 2: Search.
1 Function: SEARCH (S,
S
!
A, L, C, C :
):
a = CONTEXT(OA, )
depth = LENGTH(SPLIT-INTO-LIST(, :))`_a
nodes = GET-ALL-NODES-AT-DEPTH(depth, )C
foreach node nodes do
if node .CDN a= then
W = GET-PLANS(node)
return W
9 Function CONTEXT(object_classes, A) :
LS = SEQ-CLASS-MODEL(object_classes)
a=+`_
LS
:
foreach ( ,)! do
if L LSR
LC ARR
! then
foreach labe Cl ! do
if labe Cl ! R and label ! aY then
aa label `_=+ +
:
return a
18 Function GET-PLANS(node):
19
20
21
22
23
24
while parent 4=Y do
if parent . NW 4=Y then
WW parent . N
=+ W
return W
if node . NW 4=Y thenWW= node . N
parent = node.PN
model (see the " Sequence Classification Model " section) for
sentence embedding [36].
This embedding enables us to evaluate the similarity
among vectors by computing their cosine similarity, a measure
that tells us how alike their syntactic contexts are:
v
v
cos() =
HW6 !IP
P
N
N
v
IS
IS
P $
v
O
O N
,
(9)
Slice
Turn On/Off
(0, 0, 0, 0)
Slice
(0, 0, 0, 0) (0, 0, 0, 0)
Slice
(0, 0, 0, 0)
CDN = " locality "
CDN = " "
IEEE Robotics & Automation Magazine - March 2022

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - March 2022
