IEEE Consumer Electronics Magazine - May 2018 - 27

shown in Figure 4, the proposed system constantly processes
the incoming frames.
Let us again take LDB as an example. From the 0th to
the 48th frames, without tracking any objects, our proposed
scheme processes frames quickly. After the user selects the
first object, in the 48th frame, the system starts to track the
object and begins the corresponding RIS job, which runs in
the background from the 49th to the 59th frame. At the
98th frame, the user selects another object, which causes
the system to track two objects at the same time; the corresponding RIS job runs in the background from the 99th to
the 106th frame. From the 137th to the 174th frame, three
objects are tracked at the same time. From Table 2, one can
clearly see that the RIS is extremely time consuming.
When tracking three objects simultaneously, the RIS runtime is almost six times that of AR tracking. Without AR
tracking, the RIS would have to be run for each frame to
locate the food and to overlay the nutritional information
over the food item. As a result of this significant delay, the
system would be inefficient. By contrast, with AR tracking,
the RIS needs to be run only once in the background, and
the proposed system can remember the associated dietary
information and update the coordinates by tracking the
food between frames.
In addition, the performance of LDB and ORB is comparable in real mobile devices, since they both show a trend of
increasing runtime. Specifically, in the case of tracking one
object, the difference between LDB and ORB is small. The
average frame processing time is 3.94 s with the ORB
descriptor and 3.91 s with the LDB. Furthermore, 7.73 s and
10.82 s are needed for each frame with ORB in the case of
tracking two objects and three objects, respectively, while
LDB is slightly better than ORB, needing average times of
7.29 s and 10.45 s, respectively. Notice that the RIS runtime
is actually different for each subsequent food item because
of variances of network latency and image complexity. In
addition, the RIS runtime here actually includes the time for
accessing the remote database.

FURTHER TRACKING ANALYSIS
To further explore the tracking performance of the LDB and
ORB descriptors, we compared the computation times of several components of the processing that occurs for each frame,
including runtimes for descriptor construction and matching.
For each incoming frame, we computed 500 descriptors, and
for each descriptor we searched for its nearest neighbor in the

Table 1. The recognition rate of the RIS.
Number

Food

Recognition Rate
Supermarket
Scene

FIDS30
Database

Apple

93%

100%

Banana

100%

Blackberry

73%

80%

Cantaloupe

33%

73%

Grape

60%

80%

Tomato

87%

Kiwi

87%

93%

Orange

80%

93%

Lemon

73%

80%

Avocado

73%

93%

75.9%

87.9%

Average

1,200
Zero
e o Objec
Objects
s
RIS 1
One Object
RIS 2
Two Objects
RIS 3
Three Objects

Overall Runtime (s)

1,000
800
600
400

ORB
LDB

200
0

150
100
Frame Number

200

250

FIGURE 4. The overall speed testing under two different descriptor implementations, ORB and LDB. The light and dark red line
segments represent the time period in which one object was
tracked. The light and dark blue line segments represent the time
period in which two objects were tracked. The light and dark purple line segments represent the time period in which three
objects were tracked. Meanwhile, one RIS job runs in the background during each dark segment and for one food item once.

Table 2. The average runtime per frame.
Speed Testing

One Object

Two Objects

Three Objects

Average AR tracking runtime per frame (ORB)

3.94546

7.73204

10.82281

Average AR tracking runtime per frame (LDB)

3.91106

7.29658

10.45373

RIS runtime

66.8139

MAy 2018

IEEE Consumer Electronics Magazine

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - May 2018