IEEE Robotics & Automation Magazine - September 2017 - 135

Conclusions and Future Work
During recent decades, robotics research moved from stationary robotic systems in constrained environments to mobile
and service-oriented robots operating in realistic and unconstrained environments. One rising application field is assistive
robotics, aimed at developing robots that support humans as
their daily-life assistants. With that aim, these systems must
be endowed with different abilities, such as localization, mapping, path planning, obstacle avoidance, object detection, recognition, and manipulation.

Parallel Speedup Versus Image Size

10,000
Speedup

1,000
100
10

0.1

32 × 24
64 × 48
96 × 72
128 × 76
160 × 120
192 × 144
256 × 192
320 × 240
384 × 288
448 × 336
512 × 384
576 × 432
640 × 480
768 × 576
960 × 720
1,280 × 960
1,600 × 1,200

Image Resolution
Figure 11. The speedup versus image size for parallel (GPU) and
nonparallel (CPU) computing.

1,000

Speedup

much slower, growth. This is apparent in Figure 11, which
plots the speedup with respect to image size. In fact, the execution time for the GPU remained virtually constant
(around 0.48 s) for the first ten image resolutions considered
because the thread loads remained similar. Given that the
number of threads was limited, when the image resolution
was increased, both the thread workload and, consequently,
the execution time rose, resulting in 0.95 s for our higher
resolution (1,600 × 1,200).
Another key issue in practical object recognition is that of
scalability, and our last experiment analyzed the execution
time when the number of potential target objects was
increased. With that aim, different image sequences from the
RGB-D image data set were used. The results, shown in
Figure 12, illustrate the speedup evolution for an averaged
image resolution of 84 × 85 pixels when the number of objects
that could be found in the scene increased. As can be
observed, our results highlight the efficiency when parallel
computing is used; computation times remained almost
unchanged between one object (0.46 s) and 50 target objects
(0.47 s). Keeping in mind our final goal, an autonomous assistive robot, the system should provide a similar response time
regardless of the task at hand, as was the case, and, ideally, this
response time should be the same as that of human beings. As
our results show, the obtained response time was similar in all
the studied cases (up to 50 target objects) and below 0.5 s,
approximately twice the average human reaction time
(between 200 and 250 ms [88]-[90]). In the context of
human-computer interaction [91]-[93], a response time
below 0.1 s is regarded as an instantaneous reaction, whereas a
response delay between 0.1 and 1.0 s is considered as fast
enough for a fluent interaction, even though the user would
notice the delay. Consequently, a response time of 0.5 s is a
real-time performance in this sense. In fact, with this implementation, real-time processing could be obtained even when
hundreds of object instances are searched, taking us closer to
the possibly thousands of objects that could be found in
everyday life.
On the other hand, advances in image technology are
leading to visual sensors with higher image quality to the
effect that higher and higher image resolutions can be expected in the future. For resolutions higher than 1,600 × 1,200,
execution times would be presumably beyond 1.0 s. In this
case, image resolution could be decreased by using, for
instance, pyramidal images to obtain real-time performance.

Parallel Speedup Versus Number of Objects

100

10
15
25
35
Number of Target Objects

Figure 12. The speedup versus the number of potential target
objects for parallel (GPU) and nonparallel (CPU) computing.

In this article, we focused on object detection and recognition. Even though this issue is the heart of different robotic
assistive abilities, real-time efficient object detection and recognition is still a challenging problem when real scenarios are
considered. Part of this problem is due to the presence of cluttered, dynamic backgrounds, with possible occlusions, interactions, and additional photometric and geometric variations.
Motivated by these challenges, we presented a framework
that is able to detect and recognize objects from a visual input
in unconstrained scenes in real time. We took inspiration from
biology and used a rich object description based on color,
motion, and shape cues. Robust color information was
obtained thanks to an adequate color model choice that made
visual data invariant to changes in viewpoint, object geometry,
and illumination. The second considered cue was motion,
which was perceived by means of a novel background maintenance technique overcoming the environmental constraints of
existing methods. Finally, a phase-based representation of
shape concluded the object description presented in this article.
Once the visual features were properly extracted, the system analyzed the statistical similarity between the detected
SEPTEMBER 2017

IEEE ROBOTICS & AUTOMATION MAGAZINE

135

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2017