IEEE Circuits and Systems Magazine - Q2 2021 - 55

Rahnama et al. [80] first implement and evaluate
an FPGA accelerated adaptation of the ELAS algorithm,
which achieved a frame rate of 47 fps (up to
30× compared high-end CPU) while consuming under
4 W of power. By taking advantage of different
components on the SoC, several elaboration blocks
such as feature extraction and dense matching are
executed on FPGA, while I/O and other conditional/
sequential blocks are executed on ARM-core CPU.
The authors also reveal the strategy to accelerate
complex and computationally diverse algorithms
for low power and real-time systems by collaboratively
utilizing different compute components. Later,
by leveraging and combining the best features
of SGM and ELAS-based methods, Rahnama et al.
[81] propose a sophisticated stereo approach and
achieve an 8.7% error rate on the challenging KITTI
2015 dataset at over 50 fps, with a power consumption
of only 4.5 W.
F. CNN-Based Stereo Vision System on FPGA
Convolutional neural networks (CNNs) have been
demonstrated to perform very well on many vision
tasks such as image classification, object detection,
and semantic segmentation. Recently, CNN has
also been utilized in stereo estimation [91], [92] and
stereo matching [93]. CNN is applied to determine
SGM penalties [94], estimate real-time optical flow
disparity [95] and predict cost volume computation
and aggregation [96].
CNN has been deployed on FPGA platforms in several
works [97]-[100], with an example of lightweight YOLOv2
for object detection [101]. Nakahara et al. implement
a pipelined-based architecture for lightweight
YOLOv2 with a binarized CNN on Xilinx ZCU102 FPGA
platform. This design achieves a 40.81 fps object detection
speed, which is 177.4× faster than ARM CortexA57
and 27.5× faster than NVIDIA Pascal embedded
GPU. Many FPGA-based CNN accelerator implementations
have been summarized in [15].
IV. Localization on FPGA
A. Overview
For robots, one of the most critical tasks is localization
and mapping. Simultaneous Localization and Mapping
(SLAM) is an advanced robot navigation algorithm for
constructing or updating a map of unknown surroundings
while simultaneously keeping tracking the robot's
location. Localization and mapping are two concurrent
tasks and cannot be solved independently from
each other. Localizing a robot requires a sufficiently
detailed map, and constructing or updating or a map
SECOND QUARTER 2021
IEEE CIRCUITS AND SYSTEMS MAGAZINE
55
Table II.
A comparison between different designs on performance (MDE/s) and accuracy results on Middlebury Benchmark.
(The lower of average bad pixel rate means the better stereo matching performance.)
Reference
Shan et al. [88]
Shan et al. [89]
Wang et al. [75]
Jin et al. [69]
Jin et al. [66]
Zhang et al. [67]
Banz et al. [74]
Jin et al. [72]
MDE/s
nonocc1
15437 -
13076
10472
9362
4522
3020
1455
590
3.62
2.39
1.66
9.79
3.84
4.1
1.43
Tsukuba
all2
Venus
disc3
24.5 -
4.15
3.27
2.17
11.6
4.34
-
2.51
1nonocc: average percentage of bad pixels in non-occluded regions.
2all: average percentage of bad pixels in all regions.
3disc: average percentage of bad pixels in discontinuous regions.
14.0
8.87
7.64
20.3
14.2
-
6.6
nonocc
-
0.48
0.38
0.4
3.59
1.2
2.7
2.37
all
disc
15.7 -
0.87
0.89
0.6
5.27
1.68
-
2.97
2.79
1.92
1.95
36.8
5.62
-
13.1
Teddy
nonocc all
-
7.54
6.08
6.79
12.5
7.17
disc
15.1 -
14.7
12.1
12.4
21.5
12.6
11.4 - -
8.11
13.6
19.4
15.4
17.1
30.6
17.4
15.5
nonocc
-
3.51
2.12
3.34
7.34
5.41
8.4
8.12
Cones
all
Average Bad
Pixel Rate
disc
14.1 -
11.1
7.74
8.97
17.6
11.0
- -
13.8
9.64
6.19
9.62
21.0
13.9
16.4
all = 17.3
7.65
5.61
6.05
17.2
8.2
nonocc = 6.7
8.71

IEEE Circuits and Systems Magazine - Q2 2021

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q2 2021