IEEE Robotics & Automation Magazine - December 2019 - 26

First, the robot plans and executes its footsteps to
approach the workbench [Figure 5(a)]. It, then, detects the
plasterboard using an image and a point cloud and computes its position with respect to the plasterboard. If the
position is out of the acceptable region, the robot steps to
adjust it. Once its standing position is in the acceptable
region, the robot places its hands on the plasterboard and
approaches the workbench. Then, it leans over the workbench, stretches its arms forward, hooks its fingers to the
edge of the board, and slides the sheet [Figure 5(b)]. After
returning to an upright posture and stepping back, it pulls
out the board [Figure 5(c)]. The robot rotates the board by
pushing down on the nearest edge with its right hand and
holding it with both arms [Figure 5(d)]. Then, the robot
Wall Under Construction

Stacked Plasterboard Sheets

Screwing Tool

Workbench

Figure 4. The mock-up of a house construction site.

picks up the board and rotates it 90° [Figure 5(e)]. Owing to
its high-power joints and arm configuration, the HRP-5P
can handle large and heavy objects, such as the plasterboard.
While picking up and rotating the board, the robot must
grasp the sheet by hooking its fingertips because it has a kinematic limitation. To grasp the board firmly by cramping it
with all of its fingers, the robot puts the board down and picks
it back up before carrying it to the wall. While carrying the
board, the robot's field of view is obstructed, and its feet slip
because of the heavy load. Even in that situation, the robot
can plan and adjust its footsteps so that it can reach the
expected position by localizing its position in its memorized
3D map [20] [Figure 5(g)].
Figures 6 and 7 present snapshots of RViz and Choreonoid, respectively, taken when the robot begins to carry the
board. The dark yellow region in Figure 6 shows the floor
area where the robot can walk, and the 3D map is displayed
by voxels colored according to their height. The object-detection results are shown at the top, and an image obtained from
the head camera is shown at the lower right. Figure 7 also
shows the user interface of the task-sequencer system, which
was used to operate the robot manually when the automatic
execution stopped because of errors.
After placing the board on the floor, the robot picks up a
screwing tool from a box with its right hand while pushing on
the board with its left hand [Figure 5(h)]. The robot measures
the tool's position by detecting the LentiMark on the implement
through a camera in its right-hand palm [Figure 5(i)]. Since the
board is not fixed yet, the robot inserts screws at the middle
height while applying pressure to the board [Figure 5(j)]. Then,

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

(i)

(j)

(k)

(l)

Figure 5. Snapshots of the plasterboard installation. (a) Approach the workbench. (b) Slide the board on top. (c) Pull out the board.
(d) Push down and rotate the board. (e) Pick up and rotate the board. (f) Regrasp and carry the board. (g) Put down the board. (h)
Push the board with the left hand. (i) Pick up the tool. (j) Screw the middle position. (k) Screw the top corner. (l) Screw the bottom
corner.

26

*

IEEE ROBOTICS & AUTOMATION MAGAZINE

*

DECEMBER 2019
IEEE Robotics & Automation Magazine - December 2019

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2019
