IEEE Robotics & Automation Magazine - December 2022 - 37

Introduction
The last few years indicated fast technological improvements
in the artificial intelligence of robots. The International
Federation of Robotics records a market growth of
12% in 2020 of professional robots used for various applications,
such as transport, inspection, cleaning, medical, or
hospitality [1]. The report forecasts exponential growth for
the upcoming years. Robots will become a part of human
society. Acting side by side and collaborating within the
same environments motivate robot perception being consistent
with human perception.
In the past, robots mainly fulfilled highly customized tasks
in industrial applications independently and separated from
humans [2]. People adapted the environments to the application
requirements. However, adaptations for the robots' needs
are especially undesirable in nonindustrial environments.
Current products on the market mirror this issue through
specialized applications. These products either fulfill a single
task autonomously, such as vacuum cleaners and lawnmower
robots executing their routine independently from humans,
or interact with humans physically with a low autonomy level
[1]. Therefore, special attention must be directed to scene perception
as enabling technology to break this tradeoff between
robot proximity to humans and autonomy. Already in 1993,
the psychologist Ulric Neisser mentioned that " without perception
there is no knowledge " [3].
The goal of holistic scene perception
is to understand the scene by
" considering the geometric and
semantic context of its contents and
the intrinsic relationships between
them " [4]. Scene perception is making
sense of real-world scenes as a whole
by enabling the interaction with the
scene [5]. On the one hand, it offers
new opportunities for multiuse applications
regarding the cost per function
and physical assistance systems. On
the other hand, it increases autonomous
capabilities by overcoming challenges,
such as the occlusion of objects;
instance-specific handling of dynamics;
or spatial-temporal reasoning of
complex situations. Holistic scene perception
will enable safe and complex
behaviors for any collaboration.
Increasing attention has been given
to specific areas of perception. The
approaches achieve excellent results
in semantic extraction, such as object
detection or image classification [6].
However, in the real world, it is not
sufficient that robots solely understand
single pieces of the environment.
Much more, various scene
information needs to be understood
concurrently in the context to reason within dynamics,
uncertainties, and incompleteness for high-level control
[7, Ch. 23]. Little is known about holistic scene perception,
aiming to understand the scene in this integrated manner
needed for a large spectrum of real-world applications.
Motivated by the performance of human perception, we
distanced ourselves in this research from robotic approaches
and studied theories and experiments on human perception.
Neuroscience research has been extensively studied
since Potter [8] and Biederman [9] in the 1970s. It has
reached international bandwidth since the 2000s [10]. The
latest research provides a comprehensive overview of
human scene perception [5], [10]. This article transfers the
core theories of psychological studies on human scene perception
to robots and thereby reveals similarities and differences.
The comparison of human perception with artificial
perception is not new [11], [12], [13]. However, we will
focus on the scene perception as a whole using a top-down
view of the robots' status quo.
Methodology
Figure 1 describes the methodology of this study. First, we analyzed
current mobile robot applications where we identified
a trend toward robots in everyday life, covering multiple use
cases. For future applications, scene perception will play a
Initial Problem
Identificationof Real-World Challenges in
Robotic Scene Perception
Human Scene Perception
Literature Survey On Psychological
Theories On Human Perception
Robot Scene Perception
Literature Survey On Robot
Perception Methods
III. Transfer Human Scene Perception to Robots
Step-By-Step Transfer of the Human Scene
Perception Process to Mobile Robots
IV. Status Quo
Identify the
V. Gaps
Identify Gaps
Coverage of Holistic Scene
Perception Approaches
in Scene Perception
of Robots
VI. Potentials
Identify Potentials of
Robots Beyond
Human Capabilities
VII. Future Directions
Addressing Key Technologies for Future Research
Figure 1. A methodological approach with section references. III: The " Transferring
Human Scene Perception to Mobile Robots " section. IV: The " The Status Quo of Holistic
Scene Perception " section. V: The " The Gap to Human Scene Perception " section. VI: The
" The Potentials of Robots " section. VII: The " Future Directions " section.
DECEMBER 2022 * IEEE ROBOTICS & AUTOMATION MAGAZINE *
37
IEEE Robotics & Automation Magazine - December 2022

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2022
