IEEE Robotics & Automation Magazine - June 2023 - 105

(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
(k)
(l)
FIGURE 8. The assembly process. (a) Initial state. (b) Move base part to the assembly workspace. (c) Wooden pin assembly. (d) Bracket
assembly. (e) Frame assembly. (f) Wooden pin assembly. (g) Frame assembly. (h) Upper screw bolt assembly. (i) Lower screw bolt
assembly. (j) Erect chair. (k) Fit seat frame. (l) Assembly is complete.
parts necessary for each step. When the gripper grips the
connector parts, it moves only by a certain depth to the connector
parts plate. However, it is necessary to estimate the 2D
position and orientation of the connector parts. To this end,
the 2D positions and orientations of the connector parts are
estimated using the camera mounted atop the assembly table
[15]. The camera coordinate system was converted to the
workspace coordinate system by using projective transformation
so that the robot could grip the connector parts recognized
by the camera. For projective transformation, ArUco
markers [16] were installed on the outer corner of the connector
part plate. The image was transformed to a size proportional
to the actual distance based on the position of each
marker. Then, by using the semantic segmentation algorithm
mask regions with convolutional neural network (Mask
R-CNN) [17], the connector parts required for each step were
identified, and the position and orientation of connector parts
were estimated.
The gripping position is the center point of each connector
part mask, which is the output of Mask R-CNN.
Because the image pixel coordinates in the projective
transformed image are proportional to the workspace coordinate
system, the gripping position can be used easily in
the robot coordinate system. The connector parts, such as
wooden pins and screw bolts, used in the furniture assembly
are small and long. In addition, because the screw bolt
is divided into the head and the body, the robot arm must be
aligned in the correct gripping direction to hold the screw
bolt and then perform the fastening operation. A principal
component analysis (PCA) was performed to estimate the
orientation of the connector parts. If the principal axis vector
was estimated by applying a PCA to the mask of each
connector part, the shape characteristics of the connector
parts were reflected, and the proper gripping orientation
could be estimated.
By selecting a connector part that the gripper could grip easily,
we were able to reduce assembly failures among the several
scattered connector parts. To increase the gripping success
rate, we applied the connector part selection method. The selection
method started with selection of the part closest to the robot
RGB Camera
and Lighting
55 cm
45 cm
FIGURE 9. The environment setup for the vision camera and
connector part plate.
JUNE 2023 IEEE ROBOTICS & AUTOMATION MAGAZINE
105
IEEE Robotics & Automation Magazine - June 2023

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2023
