IEEE Robotics & Automation Magazine - June 2020 - 156

features, but it is possible to use either the CPU or the GPU for
that purpose with the classes cv::DescriptorMatcher
and cv::cuda::DescriptorMatcher, respectively.
In the third example, we compute the dense optical flow
with the Farneback algorithm [10]. The source code for the
CPU version is as follows:
1
2
3
4
5

cv::Mat src, dst;
cv::Mat prev, cv::Mat next;
cv::Mat flow(prev.size(), CV_32FC2);
cv::cvtColor(src, next, cv::COLOR_BGR2GRAY);
cv::calcOpticalFlowFarneback(prev, next, flow, 0.5, 3, 15, 3,
5, 1.2, 0);

Since optical flow is computed with the difference between
the current and previous frames, we need to define additional variables in line 2
to store the frames. We
also define a matrix of
CUDA and other computing
float numbers f l o w
for the result [in line 3,
frameworks have become
C V _ 3 2 F C 2 means a
programming standards for 2-channel (complex) floating-point array]. This flow
matrix contains the graparallel computing.
dient of the movement
between two frames; for
each pixel location in the
original frame, the channels contain dx and dy , so that
prev_x + dx = next_x, and prev_y + dy = next_y.
The computation steps are quite simple: the color image is
converted into a gray image (line 4), and the optical flow

Figure 6. The output image of the dense optical flow algorithm; the
hue represents the flow angle, and the intensity is proportional to
the flow magnitude.

algorithm is executed (line 5). For the sake of simplicity, we
have omitted additional instructions for displaying the result
and storing the frames.
The GPU version is not very different:
1
2
3
4
5

cv::Mat src, dst, flow;
cv::cuda::GpuMat gpu_src, gpu_flow;
cv::cuda::GpuMat prev, next;
gpu_src.upload(src);
cv::Ptr fof =
cv::cuda::FarnebackOpticalFlow::create();
6 cv::cuda::cvtColor( gpu_src, next, cv::COLOR_BGR2GRAY );
7 fof->calc( prev, next, gpu_flow );
8 gpu_flow.download( flow );

Besides defining all of the intermediate matrices in GPU
memory (lines 2-3), the main difference is in the interface to
the optical flow algorithm. In this version, the algorithm
object is first defined in line 5, and then applied to the frames
in line 7. Finally, the result is downloaded to CPU memory
for visualization.
The output of the optical flow algorithm is displayed in
Figure 6. The hue of each pixel block represents the orientation of the optical flow vector at that point, and the intensity
is proportional to the magnitude of the flow. The results are
shown in Table 4. As in the previous example, the execution
times for the GPUs are lower than for the CPUs, since computing dense optical flow is a demanding operation.
Finally, we test the DNN module for OpenCV. Since version 3.1, there is a DNN module in the library that implements forward pass (inferencing) with networks pretrained
using some popular deep-learning frameworks such as
Caffe [11] or TensorFlow [12]. A backend for CUDA was
added in OpenCV 4.2.0. In this example, we use the YOLO
v3 network [13], a state-of-the-art, real-time object-detection system.
While the details of the OpenCV DNN module are
beyond the scope of this article, its design is based on a
unique interface that runs on different backends and computation devices (CPU, OpenCL, and CUDA). Consequently,
the source code is exactly the same, no matter if the CPU or
GPU is used, except for the parameters that select the appropriate backend and computation target. The values for using
the CPU are as follows:
net.setPreferableBackend(cv.dnn.DNN_BACKEND_OPENCV);
net.setPreferableTarget (cv.dnn.DNN_TARGET_CPU);

The GPU can be selected with
Table 4. The computation times (in milliseconds)
for the dense optical flow algorithm (lower, in
bold, is better).
Desktop PC

Laptop PC

Embedded PC

CPU 196.382 ± 0.889 228.838 ± 6.736 983.943 ± 12.776
GPU 33.970 ± 0.231

156

*

78.561 ± 0.498

IEEE ROBOTICS & AUTOMATION MAGAZINE

*

net.setPreferableBackend(cv.dnn.DNN_BACKEND_CUDA);
net.setPreferableTarget (cv.dnn.DNN_TARGET_CUDA);

686.960 ± 9.729

JUNE 2020

A typical output image from the DNN module is shown in
Figure 7, where several cars are correctly identified in the
input image. The frame rate for the CPU and GPU versions
IEEE Robotics & Automation Magazine - June 2020

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2020
