IEEE Robotics & Automation Magazine - March 2022 - 31

manipulation capabilities: pick and place, open and close,
turn on/off, and slice. Regarding the plans included in the
first hierarchy level (condition = " locality " in Figure 4),
these are entries included in the data set's testing split that
do not provide a GP
component (3). Therefore, we do not
further consider these plans.
Figure 9 presents a 2D T-distributed stochastic neighbor
embedding projection of the word vectors to illustrate our
word representation model. Notice that objects found in the
same locality are closer to one another compared to objects
from different places. Words such as teddy_bear and scrub_
brush have low frequencies. Therefore, they are poorly represented
in the embeddings. Similarly, words such as shelf and
floor are distributed among clusters because they appear in
multiple contexts. Figure 10(a) shows that the computational
time for the ranking stage of (AP)2 is proportional to the
number of relevant plans provided by the search stage ().W
For this plot, the sentence embedding is precomputed for all
plans in the data set, stored in memory during initialization,
and retrieved using the schemes' associated Pid
at runtime.
To report the computational saving associated with the possibility
that (AP)2 has to dynamically update potential goal
states rather than having to consider the entire data set, we
first measured two cases to choose from as the longest possible
time it would take the algorithm to compute W from the
search tree, i.e., the worst-case scenario: 1) the branch with
the deepest node in the tree ()
the highest number of plans (). These times were then
processing time associated with other stages of the algorithm
(83 ms) were considered negligible. Thus, we used these values
to represent the computational expenses associated with
planner use.
Methods such as those in [42]-[44] need to call the
planner twice for every possible goal and cannot filter
those objectives. This means that, for those methods, the
time to present options to the user would be at least
46 .. .
22 92 4s More recent approaches, such as the one
) =
in [10], avoid calling the planner twice. However, they still
cannot consider relevant goal states alone, meaning that it
would take methods such as the one in [10] at least 46.2 s
to provide options to the user. By providing means for
dichotomizing the search space, our method takes 24.7 s
to present these options, a reduction of at least 47%.
To understand the effect of such dichotomization, we
compared the number of options obtained as output with
and without dichotomizing the search space for our bedroom
setup. The results are in the third column of Table 1.
By considering only the plans in
S =
rithm missed 32 feasible options from Ll _.ivingroom
Lb ,edroom our algoS
=
59 sn and 2) the branch with
36 sn
added to what it takes (AP)2 to complete the ranking stage
for the plans referenced by their corresponding nodes and all
their ancestors in either case: 2,951 and 3,440 files, respectively.
The longest time was 83 ms, while the same process
took 0.18 s for the entire data set, resulting in an associated
improvement of at least 50%.
Due to the fundamentally different
approaches to proactive planning used
by our method (affordance based) and
other state-of-art methods (action oriented),
it is difficult to provide a direct
comparison of these methods. However,
as some of these methods leverage
recognition as planning approaches
[10] and therefore rely on a planning
system, we can provide a comparison
in terms of the computational savings
associated with planner use. Table 1
delivers this comparison. In the first
column, the table shows the number
of plans per locality in the data set
that can be considered for a robot
with navigation and pick/place capabilities.
The second column shows the
time it takes our algorithm to present
options to the user (if any). As evident,
the values in this column are on
the order of seconds. Therefore, the
20
40
60
-60
-40
-20
-80
-60
However, a manual inspection of those 32 options revealed
that they involved the same combinations of low-level
actions and objects that result from the 107 alternatives provided
by our algorithm.
For Figure 10(b), we measured the time it took the AR
HMD UI to receive options after updating the state knowledge
with an increasing number of files analyzed by the planner.
By configuring the number of plans that are input to the
feasibility filter, we can adjust this time to a suitable value for
the user, which will vary depending on his or her preferences,
thus potentially improving interactivity. To configure the
time to provide options to the user, we do not constrain the
PDDL-based planning system in any way. We instead reduce
Tomato_Sliced
Potato
Microwave
Pan
Mug
Bowl
Coffee_Machine
Sink
Plate
Vase
Dish_Sponge
Scrub Brush
Drawer
Cloth
Spray_Bottle
Hand_Towel
Toilet
-40
-20
Shelf_Floor
Teddy_Bear
Mirror
Safe
Bed
20
40
60
Figure 9. The 2D T-distributed stochastic neighbor embedding projection of the trained
word embedding model. Objects found in the same locality are closer to one another
compared to items from different areas.
MARCH 2022 * IEEE ROBOTICS & AUTOMATION MAGAZINE *
31
Chair
Sofa
Lamp
Padding
Bathroom
Living Room
Bedroom
Kitchen
Pillow
Cell_Phone
IEEE Robotics & Automation Magazine - March 2022

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - March 2022
