IEEE Robotics & Automation Magazine - June 2020 - 12

in a service robot scenario. Over 150
registered participants representing
eight teams competed in the final testing
phase. The work paved the way for
robots to behave like humans in terms of
knowledge transfer, association, and
combination capabilities.
Challenge Data Set
The data set Lifelong Robotic Vision
(OpenLORIS)-Object Recognition
(OpenLORIS-Object) is designed to
drive lifelong learning research and
potential applications in the robotic
vision domain, with everyday objects that
exist in home, office, campus, and mall
scenarios. The data set explicitly quanti-
fies the variants of illumination, object
occlusion, object size, camera-object dis-
tance/angles, and clutter information.
The IROS 2019 competition organizers
provided the first version of the OpenLO-
RIS-Object data set for the participants.

Note that our data set has been updated
with twice the size in content available at
the project home page [20], including
data set visualization, download instruc-
tions, and more benchmarks on state-of-
the-art lifelong learning methods [21].
The competition data set is a collec-
tion of 69 instances, including 19 catego-
ries of daily necessities objects under
seven scenes (see Table 1). For each
instance, a 17-s video (at 30 frames per
second) was recorded with a depth cam-
era delivering 260 distinguishable chosen
RGB-D frames. Four environmental fac-
tors, each with three level changes, are
considered explicitly (Table 1). The data
were divided into 12 sequential tasks by
randomly sampling from different factors
and levels. The organizers also provided a
more challenging bonus test set that was
recorded under different context back-
grounds with some deformation and
extreme view angles.

Challenge Rules
Rules are designed to quantify the
learning capability of the robotic vision
system when faced with the objects
appearing in the dynamic environ-
ments. Different from a standard com-
puter vision challenge, not only was the
overall accuracy on all tasks evaluated;
the model efficiency, including model
size, memory cost, and replay size (the
number of old task samples used for
learning new tasks; smaller is better),
was also considered (Table 2). Mean-
while, instead of directly asking the par-
ticipants to submit the prediction
results on the test data set as in standard
deep learning challenges [1], [2], the
organizers received either source or
binary codes to evaluate their whole
lifelong learning process to make a fair
comparison. The finalists' methods
were tested by the organizers on an Intel
Core i9 CPU and a Nvidia RTX 1080 Ti
graphics processing unit.
Challenge Methods and Results
The finalists and their results are sum-
marized in Table 3, with the top
result(s) in each category designated
in bold. Details such as the report,
slide, and poster of each solution can
be found on the project home page
[19]. With excellent participants and
the solutions they presented, we an-
ticipate that the resulting solutions can
help robots perform well under dy-
namic environments.
HIK_LIG Team (Champion)
● Title: Dynamic Neural Network for
Incremental Learning
● Members: Liang Ma, Jianwen Wu,
Qiaoyong Zhong, Di Xie, and Shil-
iang Pu
● Affiliation: Hikvision Research Insti-
tute, Hangzhou, China.

Figure 2. The Lifelong Robotic Vision Challenge finalists at IROS 2019.

Table 1. The details for each of the three levels of four real-life robotic vision challenges.

Level

Illumination

Occlusion (%)

Object Pixel Size

Clutter

Context

Classes

Instances

Strong

> 200 × 200

Simple

Normal

30 × 30 − 200 × 200

Normal

Home/office/
campus/mall

Weak

< 30 × 30

Complex

IEEE ROBOTICS & AUTOMATION MAGAZINE

JUNE 2020

IEEE Robotics & Automation Magazine - June 2020

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2020