IEEE Robotics & Automation Magazine - December 2022 - 153

goal and obstacle to the gap goal,
(),
iis ii=+ -
de gg o
is placed at a distance of do
and the local goal
in this
desired direction [shown in teal in
Figure 4(b)].
The inputs to the robot are angular
and linear velocities and are determined
using proportional controllers:
*
~ides i~
a
vk vmax c1
v
where ,kt
k ,v
==,
~
~max
min
k^ ^h htr max
m
(2)
and a are constant proportional
gains, ri is the current heading
of the robot, and max~ and vmax
are
the maximum angular and linear velocities,
respectively.
Temple University
The team at Temple (https://sites.
temple.edu/trail/) used a deep reinforcement
learning (DRL)-based control
policy, called DRL-velocity
obstacles (VO), originally designed for
safe and efficient navigation through
crowded dynamic environments. The
system architecture of the VO) control
policy, provided in Figure 5, is divided
into two modules: preprocessing and
the DRL network.
Preprocessing Module
Instead of directly feeding the raw sensor
data into deep neural networks like other
works, the DRL-VO control policy utilizes
preprocessed data as the network
input. There are three types of inputs that
capture different aspects of the scene:
1) Pedestrians: To track pedestrians, the
raw red-green-blue image data and
point cloud data from a ZED camera
are fed into the You Only Look Once
v3 object detector to get pedestrian
detections. These detections are
passed into a multiple hypothesis
tracker to estimate the number of
pedestrians and their kinematics (i.e.,
position and velocity). These pedestrian
kinematics are encoded into two
80 × 80 occupancy grid-style maps.
2) Scene geometry: To track the geometry,
the past 10 scans (0.5 s) of lidar
data are collected. Each lidar scan is
downsampled using a combination of
minimum pooling and average pooling,
and these downsampled lidar
DECEMBER 2022 * IEEE ROBOTICS & AUTOMATION MAGAZINE *
153
1 × 1 AdaptiveAvgPool2d
Bottleneck Blocks:
C = 256, S = 2
FC, C = 2
FC, C = 256
FC, C = 256
FC, C = 1
FC, C = 128
Bottleneck Blocks:
C = 128, S = 2
Bottleneck Blocks:
C = 64, S = 1
3x3 MaxPool2d, C = 64
3 × 3 Conv2d, C = 64
Actor
YOLOv3 and MHT
ZED Camera
Pedestrian Kinematic Maps: 2 x 80 x 80
Minimum and
Average Pooling
Hokuyo Lidar
T = 0:
T = 1:
T = 8:
T = 9:
Minimum Minimum MinimumMinimum Minimum Minimum
AverageAverage AverageAverage AverageAverage
MinimumMinimumMinimumMinimum MinimumMinimum
AverageAverage AverageAverage AverageAverage
Action:
2 × 1
Critic
MinimumMinimumMinimumMinimum MinimumMinimum
AverageAverage AverageAverage AverageAverage
MinimumMinimumMinimumMinimum MinimumMinimum
AverageAverage AverageAverage AverageAverage
Lidar Historical Map: 80 × 80
Pure Pursuit Algorithm
Goal Point
gx
gy
Subgoal Point: 2 × 1
Feature Extractor
Vx
Wz
V(s)
Value:
1 × 1
(a)
(b)
Figure 5. The system architecture of the Temple team's DRL-VO control policy. Raw sensor data from the ZED camera and Hokuyo lidar as well as the goal point are fed into a
preprocessing module to create intermediate data representations. These low-level intermediate features are fused in a feature extractor network to obtain high-level abstract
features. The actor network uses these abstract features to generate steering commands to control the robot, while the critic network outputs the state value for training the
policy. The (a) preprocessing module and (b) DRL network module early fusion architecture. YOLO: You Only Look Once; MHT: multiple hypothesis tracker.
https://sites.temple.edu/trail https://sites.temple.edu/trail
IEEE Robotics & Automation Magazine - December 2022

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2022
