IEEE Robotics & Automation Magazine - March 2022 - 26

{, ,, }
CC CPP Pn01 f 3 is a set of labels representing the
C
robot capabilities required to complete the plan. Notice the
ordering of the elements in (4), where LP
is followed by robot
capability labels ordered as they appear in C and that all labels
are separated by colons. We use CD to dichotomize the
search space, as we describe in the " Search " section. We
applied standard NLP cleaning and normalization methods
to extract the object classes mentioned in IP
in N .IP
We used the extracted object classes to train a word
representation model for word embedding, as we describe
in the " Search " section. After incorporating these new fields,
we express each augmented data set entry ()E+
as
EE,, N H.
+ =G CD IP
(5)
Search
We designed a search tree ()C that organizes U based on
the environmental features discussed in the previous section.
The search tree provides means for considering only
Algorithm 1: The build-search tree.
1 Function BUILD-TREE (
L, )C :
2
3
4
5
6
9
12
13
14
17
18
19
20
21
22
23
26
27
28
29
30
33
34
35
36
37
38
C = TREE-NODE( " root " )
depth = 0
ADD-LOC-NODES( ,C L, depth)
ADD-CAP-NODES( ,C C, depth)
return C
7 Function TREE-NODE(id):
8
NN idPCD
return N
== = == =
G
id NN NN
4 K
`_ W
10 Function ADD-LOC-NODES ( ,C L, depth):
11
foreach labe Ll ! do
ADD-CHILD( ,C label)
depth += 1
return
15 Function ADD-CAP-NODES ( ,C C, depth):
16
if depth = 1 then
foreach node ! C do
foreach labe Cl ! do
.KN
foreach {()}C
ADD-LEVEL(, C,C i)
depth += 1
;
return
24 Function ADD-CHILD(parent, label):
25
child = TREE-NODE(label)
chil .dPN
= parent
APPEND(child to parent.K )N
if parent.CDN =`_ then child.CDN
else
return
31 Function ADD-LEVEL (node, C, depth):
32
if depth = 0 then
foreach labelc ci { 6
XnodeN inid
INDE (. )}C
else
!
!! 2;
do
ii C
ADD-CHILD(node, label)
foreach child node N
return
.K do
ADD-LEVEL(, C,C depth-1)
= label
child.CDN = parent.CDN + " : " + label
ADD-CHILD(node, label)
ii depthi LENGTH!# #
do
,, {},, {}H
where L LS
!
relevant plans during the following stages of the algorithm,
thus improving the computational throughput.
We represent each tree node (N) in C with the following
data structure:
NNid,, K ,, ,PCDNN NN
=G
where Nid
and store them its parent node, KN
W H
(6)
is
is the name of the node in the state space, PN
is the set of children nodes of N, CDN
is the primary criteria for the expansion of the node during
a search (the node condition), and N
tifiers such that
W is a set of plan idenWU
(7)
(AP)2 builds the search tree ()C during initialization. An
NN
id
== + {} .;!6
PCDCDE
overview of the method is presented in Algorithm 1. When
the build process finishes, the algorithm incorporates the
unique plan identifier ()Pid
()E+
in the data set of plans ()U to each tree node (N) in C
by comparing the context declarations (CD) in E+
node condition (),CDN
as in (7). An example of a search tree
built from context declarations for a household environment
is given in Figure 4. The search through C begins after every
update of the current state (S), and it is performed by comparing
the current context of the user (),a which is jointly
obtained from S and the set of robot affordances A( ), with
the CDN
of all nodes in
a = LC CSn0
g
tion Model " section) and CCn
C . We express a as
:: :,
(8)
is a locality class (see the " Sequence ClassificaC
{,
,}f 3 is the set of robot
capabilities that can be applied in this locality. The output of the
search is a subset of plan identifiers that might be relevant for
the user given the context. We express this set of relevant plans
as .W For example, if a = " kitchen:navigation:pickandplace, "
only 440 + 0 + 475 = 915 plan identifiers will be provided as
output (see Figure 4). Algorithm 2 describes the method we
follow to obtain a and
W .
Sequence Classification Model
Human environments are unstructured; we cannot know in
advance how many objects will be found and where they will
be placed. However, most people will correctly identify a
bathroom after seeing a toilet and sink even if a magazine and
a remote are there, too. Given this human understanding of
the semantic distances between objects, which are included in
data sets such as those discussed in the " Data Set " section, we
chose to represent semantic distances by using a sequence
classification model. We employ it to infer where the user is
located based on the set of objects identified in the environment
O(
S). The model is composed of an embedding layer, a
recurrent neural network (RNN), and a dense layer with softmax
activation.
For word representation, also called embedding, classical
NLP approaches use one-hot vector representations for mapping
words to vectors of real numbers. More recently, dense
26 * IEEE ROBOTICS & AUTOMATION MAGAZINE * MARCH 2022
of all augmented data set entries
with the
IEEE Robotics & Automation Magazine - March 2022

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - March 2022
