IEEE Robotics & Automation Magazine - June 2020 - 153

processing steps are executed (lines 4-5); and finally, the resulting image is transferred from the GPU back to CPU memory
(line 6). The processing flow is depicted in Figure 3, where the
CPU and GPU memory spaces and the different processing
steps are represented.
There is an inherent overhead in the GPU processing flow
due to the transfer of the images between the CPU and GPU
memories. Such overhead can be minimized if all the processing operations are performed in the GPU and only the initial
and final images are transferred:
Toverhead = Tupload + Tdownload .
Let's define TCPU and TGPU as the computation times of
the image processing operations (cvtColor, threshold)
at the CPU and GPU, respectively. A speed gain will be
obtained if and only if
TCPU > Toverhead + TGPU .
These computation times depend mainly on two factors:
● Hardware technology of the respective boards: OpenCV is
highly optimized for CPUs with multiple cores and vector
instructions.
● Degree of parallelization of the processing algorithms: Some
vision operations may benefit more than others from the
use of multiple cores in the GPU.
Image Processing Applications
In the following, we elaborate on four examples of image processing applications [edge detection, feature extraction, optical flow, and object detection with deep neural networks
(DNNs)] that use OpenCV with a CPU and GPU in different
hardware configurations.
The first example is a simple edge detection application
with the well-known Canny algorithm [8]. The CPU version
of the application is as follows:
1
2
3
4
5

cv::Mat src, dst;
const int lowThreshold = 20;
const int ratio = 3;
const int kernel_size = 3;
cv::Mat src_gray, blurred, edges;

6 cv::cvtColor( src, src_gray, cv::COLOR_BGR2GRAY );
7 cv::blur( src_gray, blurred, cv::Size(3,3) );
8 cv::Canny( blurred, edges, lowThreshold,
lowThreshold*ratio, kernel_size );
9 src.copyTo( dst, edges );

The CUDA version is very similar, yet there are some
changes in the application programming interface of the processing functions:
1
2
3
4
5
6

cv::Mat src, dst;
const int lowThreshold = 20;
const int ratio = 3;
const int kernel_size = 3;
cv::cuda::GpuMat gpu_src, gpu_dst;
cv::cuda::GpuMat src_gray, blurred, edges;

7 gpu_src.upload( src ) ;
8 cv::Ptr blur =
cv::cuda::createBoxFilter( CV_8UC1, CV_8UC1,
cv::Size(3,3) );
9 cv::Ptr canny =
cv::cuda::createCannyEdgeDetector( lowThreshold,
lowThreshold*ratio, kernel_size );
10 cv::cuda::cvtColor( gpu_src, src_gray, cv::COLOR_BGR2GRAY );
11 blur->apply( src_gray, blurred );
12 canny->detect( blurred, edges );
13 gpu_src.copyTo( gpu_dst, edges );
14 gpu_dst.download( dst );

Now we need to define the original and destinations images both as Mat and GpuMat variables (lines 1 and 5). The
parameters are defined in
the same ways as in the
previous version (lines
Modern GPUs are highly
2-4), and the intermediate images are defined
parallel, multicore systems,
as GpuMat (line 6). The
original image is uploadpowerful enough to
ed to the GPU memory
in line 7. Then, two new
perform general purpose
objects have to be defined
for applying the blur filter
computations on large
and the Canny detector,
respectively, in lines 8 and
blocks of data.
9. Image processing is
executed in lines 10-12,
and the original image is
masked with the detected edges and copied to the destination
(line 13). Finally, in line 14 the result is downloaded to the

GPU Memory
gpu_src

src_gray
cvtColor

Besides the initial and final images defined in line 1, three
more variables are created in line 5 for storing the intermediate images. The algorithm parameters are defined in lines
2-4, and the processing steps (converting to gray, blurring,
and computing the edges) are executed in lines 6-8. Finally,
the edges are used as a pixel mask for copying the original
image to the destination image in line 9.

gpu_dst
Threshold

Upload

Download

src

dst

CPU Memory
Figure 3. The image processing flow with a CPU and GPU.

JUNE 2020

*

IEEE ROBOTICS & AUTOMATION MAGAZINE

*

153
IEEE Robotics & Automation Magazine - June 2020

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2020
