IEEE Consumer Electronics Magazine - May 2018 - 30

Recognizing different types of
meat is not entirely a solved
problem because of how similar
they appear.
ratio of the correct word and the jitter were both higher for
simple backgrounds, appearing as the set of points in the first
zone of the plot. As the background complexity increases, the
ratio becomes lower, while the tracking stability becomes
higher, appearing as the set of points in the second zone in
the plot. The points in the third zone indicate that there is a
critical line between jitter and RIS.
In this experiment, we tested the same object scene at
different distances. AR and RIS always used the same initial camera frame. We started with the shortest distance
(the simplest background) and then progressed to the longest (the most complicated background). From Figure 9,
one can clearly see this trend. As distance increased, both
jitter and the ratio fell, which means that the performance
of the RIS is somehow inversely proportional to that of
tracking. Furthermore, at shorter distances, there is a
noticeable clump in the data (the second zone of the plot),
where the ratio maintains its value while jitter decreases
quickly. At slightly farther distances, however, one can see
a critical point in the figure, where the background is sufficiently complicated to confuse the RIS without significant
reductions in jitter.

DISCUSSION
With AR tracking, the proposed system can overlay the
nutritional information on the image of the food in a stable
and relatively efficient manner. The quality of the region
of interest will affect the final RIS recognition rate. Imag-

es with a high illumination level and a pure white background have the best recognition rate, whereas those with
overcomplicated backgrounds have relatively lower such
rates. Other factors that might lower the recognition rate
include the appearance of several food items in the same
image, and an orientation of the food that differs from
what the recognition algorithms expect. Poorer identification is also the case with dark-colored food, items with a
rough surface, foods of similar color, and food that is partially occluded.
A more efficient solution would be to train food-specific
models; however, this potential approach does not scale
well. In contrast, for arbitrary food items, RIS was able to
find similar images in Google's database. Nevertheless,
some food items are simply too difficult to distinguish, even
for humans. For example, recognizing different types of
meat is not entirely a solved problem because of how similar they appear. It is necessary to further incorporate an
algorithm to find unique features of different meats. Alternatively, one could scan the barcode on the food item to
determine its name and then query for nutritional information as before.
Camera and image processing are the largest contributors to Google Glass's power consumption [11]. As with all
electronics, power is linked to temperature, and Google
Glass also ran much hotter over Wi-Fi and image processing. In terms of the runtime, Google Glass currently does
not allow graphics processing unit acceleration on a regular
Android smartphone and tablet; therefore, we cannot optimize the runtime much as things stand. However, Google
Glass is still under development, and a new generation may
appear on the consumer market in the near future. We
believe that our work can help Google engineers or those
of a related company to identify some application needs
and challenges in terms of hardware requirements and software support.

CONCLUSION

Ratio of the Correct Word

0.18
0.16
0.14
0.12
0.1
0.08

0.06

0.04
0.02
0

6
8
10 12
1/Jittering (1/Pixel)

FIGURE 9. The tradeoff between AR and RIS. The ratio is between
the number of occurrences of the correct word and the total number of words returned by the RIS, and represents the RIS performance. The 1/jittering represents the tracking stability.
30 IEEE Consumer Electronics Magazine

MAy 2018

In this article, we proposed a promising application (food
nutrition information retrieval and visualization in the
field), compared the performance differences when choosing different computational algorithms (baseline and cutting
edge), and evaluated the limitation of consumer electronics
(Google Glass) using this novel application. The proposed
food-monitoring system recognizes food, retrieves the nutritional information from a remote database, and overlays
the dietary facts on the food surface. We employed RIS to
find similar images, a blacklist and a whitelist to recognize
food items, and AR to overlay the nutritional data on the
food surface.
With two different descriptor implementations, we performed four experiments to verify the usability of the proposed system, including speed testing, the RIS recognition
rate, AR stability, and the tradeoff between jitter and RIS.
The results indicated that the proposed AR system can determine, fetch, and overlay the nutritional information on the

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - May 2018