IEEE Robotics & Automation Magazine - September 2015 - 44

for grasp synthesis and execution, while other objects have
higher shape complexity (e.g., the spring clamps in the tool category, or the spatula in the kitchen-items category) and are
more challenging for grasp synthesis and execution. Considering these aspects, the proposed set has a superior variety compared with the commercially available sets [8], [11], [41], [42],
[44], which are designed to address some particular manipulation aspects only.
● Use: We included objects that are not only interesting for
grasping but that also have a range of manipulation uses.
For example, a pitcher and
a cup; nails and a hammer; and pegs, cloths, and
Our goal is to do as much
rope. We also included assembly items/tasks: a set
as possible to facilitate
of children's stacking cups,
a toy airplane (Figure 6)
the widespread usage of
that must be assembled
and screwed together, and
a common set of objects
Lego Duplo bricks (Figure
7). In addition, widely
and tasks to allow easy
used standard manipulation tests in rehabilitation,
comparison of results
such as an improvised
box-and-blocks [41] and a
between research
nine-hole peg test [42], are
included. As mentioned
groups worldwide.
above, these tasks are intended to span a wide
range of difficulty, from relatively easy to very difficult. Furthermore, the ability to quantify the task performance was
also prioritized, including aspects such as level of difficulty,
time to completion, and success rate, among others.
Table 3. The suggestions for manipulation tasks.
Object Category Suggested Tasks

Food items

* Packing/unpacking the groceries

Kitchen items

* Table setting
* Wipe down table with sponge and
Windex
* Cooking scenarios

Tool items

* Nailing
* Drilling
* Unlocking the padlock using the key
* Placing the pegs on the rope
* Unscrewing a bolt using the wrench
* Cutting a paper with the scissors
* Writing on a paper
* Screwing the nut on the bolt

Shape items

* Sorting marbles into the plastic blocks
* Unstacking/stacking the cups
* Placing the washers onto the bolt

Task items

* Box-and-blocks test
* Toy-plane assembly/disassembly
* Nine-hole peg tests
* Lego assembly/disassembly
* Cloth folding

IEEE ROBOTICS & AUTOMATION MAGAZINE

September 2015

Durability: We aimed for objects that can be useful in the
long term, and, therefore, avoid objects that are fragile or
perishable. In addition, to increase the longevity of the object set, we chose objects that are likely to remain in circulation and change relatively little in the near future.
● Cost: We aimed to keep the cost of the object set as low as
possible to broaden accessibility. We, therefore, selected
standard consumer products, rather than, for instance, custom-fabricated objects, and tests. The current cost for the
objects is approximately US$350.
● Portability: We aimed to have an object set that fits in a
large-sized suitcase and be below the normal airline weight
limit (22 kg) to allow easy shipping and storage.
After these considerations, the final objects were selected
(Table 2 and Figures 1-8). Objects 1-18 are the food items, including real boxed and canned items as well as plastic fruits,
which have complex shapes. Objects 19-34 are kitchen items,
including objects for food preparation and serving as well as
glass cleaner and a sponge. Objects 35-52 form the tool items
category, containing not only common tools but also items-
such as nails, screws, and wood-with which to utilize the
tools. The shape items are objects 53-67, which span a range
of sizes (spheres, cups, and washers), as well as compliant objects such as foam bricks, rope, and chain. The task items are
objects 68-77, and they include two widely used tasks in rehabilitation benchmarking (box-and-blocks [41] and nine-hole
peg test [42]) as well as items for relatively simple and complex
assembly tasks (a Lego Duplo set and children's airplane toy,
respectively). Furthermore, the set includes a black T-shirt for
tasks like cloth folding as well as a magazine and a Rubik's
cube. We include a timer in the kit (Figure 8), which not only
provides accurate timing of the task but also serves as a manipulation object with a keypad. While there are an unlimited
number of manipulation tasks that might be able to be done
with these objects, we provide some examples for each category in Table 3 (with an in-depth discussion of tasks and protocols in the "Conclusions and Future Work" section).
●

Object Scans
To ease adoption across various manipulation research approaches, we collected visual data that are commonly required for grasping algorithms and generate 3-D models for use in simulation.
We used the same scanning rig used to collect the BigBIRD data
set [1]. The rig, shown in Figure 9, has five RGB-D sensors and
five high-resolution RGB cameras arranged in a quarter-circular
arc. Each object was placed on a computer-controlled turntable,
which was rotated by 3° at a time, yielding 120 turntable orientations. Together, this yields 600 RGB-D images and 600 high-resolution RGB images. The process is completely automated, and the
total collection time for each object is under 5 min.
We then used Poisson surface reconstruction to generate
watertight meshes [54] (Figure 10). Afterward, we projected
the meshes onto each image to generate segmentation masks.
Note that Poisson reconstruction fails on certain objects with
missing depth data; specifically, transparent or reflective regions of objects usually do not register depth data. We will

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2015