IEEE Robotics & Automation Magazine - December 2022 - 38

key role as it provides comprehensive scene knowledge to
the cognitive intelligence of robots. Therefore, we analyzed
state-of-the-art robot scene perception approaches and theories
of human scene perception to elaborate on a transfer.
Research in both domains offers a comprehensive overview
providing fundamental
knowledge as the starting
point. The " Transferring
Human Scene
Perception to Mobile
Robots " section summarizes
the transfer with a
step-by-step analysis of
the similarities and differences
between human
and robot scene perception.
Based on this, we
addressed the status quo
of holistic scene perception
approaches of robots
within a qualitative
evaluation (see the " The
Status Quo of Holistic Scene Perception " section) to extract
how far robotic scene perception reached the performance of
human perception to reveal gaps (see the " The Gap to Human
Scene Perception " section) and potentials (see the " The
Potentials of Robots " section). Finally, we propose future directions
for robotic scene perception (see the " Future Directions "
section).
Transferring Human Scene Perception
to Mobile Robots
Humans perceive the environment by recognizing scene
information through different senses, such as sight, hearing,
This article transfers the
core ideas from human
scene perception onto
robot scene perception to
contribute toward a holistic
scene understanding
of robots.
smell, touch, or taste. Herewith, researchers discovered that
people primarily rely on their visual perception for perceiving
the environment due to the richness of information [14].
Although the process of human perception is unknown in
detail, most theories define perception as the process of " recognizing
(being aware of), organizing (gathering and storing),
and interpreting (binding to knowledge) sensory information "
[15, Ch. 3]. We transfer these three process steps to
robot scene perception to reveal similarities and differences to
the latest robotic research (Figure 2).
The Recognition of Information
The recognition step processes sensory information to make
it understandable for the subsequent perception tasks. Using
the sensory input, humans convert the observations from the
scene into understandable information. Whereas vision benefits
from a high amount of information, other modalities,
such as haptics, are robust to noise and independent of light
exposure to get properties being insufficiently recognized by
other senses [16]. For example, the eye serves as a transducer
to convert light into the optic nerve by five layers of cells (particularly
photoreceptors) within the retina [17]. The nerve
cells transmit information by electric signals to the brain for
processing [18]. The processing itself divides into preattentive
and postattentive recognition.
In preattentive processing, activities related to low-level
Recognition
of Information
Knowledge
Representation
Knowledge
Interpretation
vision are usually associated with the extraction of certain
physical properties of the visible environment, such as
depth, shape, color, object boundaries, or surface material
properties [10], [19], [20]. Humans are not capable of processing
all sensory information at once. Therefore, the postattentive
processing extracts scene information based on
attention areas to overcome this issue [15]. It compresses
high-level features, such as object classification or redetection
[21], [22]. Herewith, humans
receive information from hierarchical
levels of abstraction [23], [24] corresponding
to a region of interest, triggered
by the attention that provides
the semantic of sensory input.
Similar to humans, the recognition
Neural Processing
Knowledge Base
Association
Scene Recognition
Database
Scene Analysis
Figure 2. The process of human perception transferred to mobile robots.
38 * IEEE ROBOTICS & AUTOMATION MAGAZINE * DECEMBER 2022
of robots can be divided into pre- and
postattentive recognition. Equivalent
to the human senses, robots use various
sensors to recognize information
from the environment. The most
common sensor modalities are visual
sensors, such as cameras, lidar, ultrasonic,
and radar. They are capable of
providing the color and/or range
information of the surrounding. The
recognition of visual features is the
most popular modality for robots
due to the information richness.
However, few approaches investigate
other modalities, such as acoustic
IEEE Robotics & Automation Magazine - December 2022

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2022
