IEEE Robotics & Automation Magazine - June 2020 - 11

COMPETITIONS

IROS 2019 Lifelong Robotic Vision:
Object Recognition Challenge
By Heechul Bae, Eoin Brophy, Rosa H.M. Chan, Baoquan Chen, Fan Feng, Gabriele Graffieti, Vidit Goel, Xinyue Hao, Hyonyoung Han,
Sathursan Kanagarajah, Somesh Kumar, Siew-Kei Lam, Tin Lun Lam, Chuanlin Lan, Qi Liu, Vincenzo Lomonaco, Liang Ma,
Davide Maltoni, German I. Parisi, Lorenzo Pellegrini, Duvindu Piyasena, Shiliang Pu, Qi She, Debdoot Sheet,
Soonyong Song, Youngsung Son, Zhengwei Wang, Tomas E. Ward, Jianwen Wu, Meiqing Wu,
Di Xie, Yangsheng Xu, Lin Yang, Qihan Yang, Qiaoyong Zhong, and Liguang Zhou

H

umans have a remarkable
ability to learn continuously
from the external environment
and inner experience. One of
the grand goals of robots is to build an
artificial "lifelong learning" agent that
can shape a cultivated understanding of
the world from the current scene and
previous knowledge via an autonomous
lifelong development. It is challenging
for the robot learning process to re-
tain earlier knowledge when robots
encounter new tasks or information.
Recent advances in computer vision
and deep-learning methods have been
impressive due to large-scale data sets,
such as ImageNet [1] and COCO [2].
However, robotic vision poses unique
new challenges for applying visual
algorithms developed from these
computer vision data sets because they
implicitly assume a fixed set of
categories and time-invariant task
distributions [3].
Semantic concepts change dynami-
cally over time [4]-[6]. For bridging
the gap between robotic vision and
stationary computer vision fields, we
utilize a real robot mounted with mul-
tiple high-resolution sensors [e.g.,
monocular/red- green-blue-depth
(RGB-D) from RealSense D435i, dual
fisheye images from RealSense T265,
and lidar; see Figure 1] to actively col-
lect the data from the real-world
objects in several kinds of typical sce-
narios, such as homes, of f ices,
campuses, and malls.
Digital Object Identifier 10.1109/MRA.2020.2987186
Date of current version: 10 June 2020

Lifelong learning approaches can be
divided into
1) regularization methods, e.g., Learn-
ing without Forgetting (LwF) [7],
elastic weight consolidation (EWC)
[8], and synaptic intelligence (SI) [9]
2) network expansion methods, e.g.,
context-dependent gating [10] and
Dynamic Expandable Network [11]
3) rehearsal approaches with a sampling
replay or generative mechanism to fit
distribution from prior tasks [12]-
[14], e.g., incremental classifier and
representation learning [15], Deep
Generative Replay (DGR) [16], and
DGR with dual memory [17] and
feedback [18].
This report summarizes the IEEE/
RSJ International Conference on Intelli-

gent Robots and Systems (IROS) 2019
Lifelong Robotic Vision Competition
(Lifelong Object Recognition Chal-
lenge) with the data set, rules, methods,
and results from the top eight finalists
(of over 150 teams) (Figure 2). Individ-
ual reports, data set information, rules,
and released source codes can be found
at the project home page [19].
Challenge Data Set and Rules
This challenge aimed to explore how to
leverage the knowledge summarized
from previous tasks for learning a new
task efficiently as well as how previously
learned tasks could be efficiently memo-
rized in lifelong robotic vision. The goal
of this competition was to test a model's
capability to continuously learn objects

z
y
x

Base_Link
(Ground-Truth Target Frame)

Laser

Marker (Centroid of OptTrack Markers)
d400_depth/d400_imu
d400_color
t265_fisheye2
t265_imu

t265_fisheye1

Figure 1. The OpenLORIS robotic platform (left) mounted with multiple sensors (right).
In the OpenLORIS-Object data set, the RGB-D data are collected from the depth
camera.

JUNE 2020

*

IEEE ROBOTICS & AUTOMATION MAGAZINE

*

11
IEEE Robotics & Automation Magazine - June 2020

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2020
