IEEE Consumer Electronics Magazine - May 2018 - 22

Displaying nutrition information
before the customer buys or eats
food can help people make
better choices and form
good eating habits.
The WHO pointed to several methods to reduce and prevent
obesity, specifically, making healthier eating choices and
engaging in regular physical activity. In relation to this, Wardle
[28] concluded that nutrition knowledge and food intake follow
demographic patterns. In other words, knowledge was significantly associated with healthy eating, and the effect persisted
after controlling for demographic variables. However, when
selecting food, most people lack nutrition facts. They may not
realize which foods contain fewer calories and greater nutrition
that their bodies require. Therefore, displaying nutrition information before the customer buys or eats food can help people
make better choices and form good eating habits [4]. The
results in [5] and [7] also showed that consumers substantially
underestimated the calorie levels of less-healthful food, but that
the preference for less-healthy items diminished when nutrition
facts were disclosed. Thus, food monitoring or nutrition visualization is valuable and can be considered as a potential way to
slow the increasing prevalence of obesity.
Traditionally, food nutrition information has been acquired
manually. This includes, but is not limited to, manual Internet
searches for various foods and relying on prior knowledge (or
avoiding the topic altogether). The effectiveness of these methods is limited: manual Internet searches are time consuming,
prior knowledge can be hazy, and ignoring the problem has
contributed to the current state of affairs. Therefore, many
researchers have made efforts to implement electronic devices
that can automatically collect food nutrition information and
record diet data. For example, Sebastian et al. conducted
experiments that required users to wear a sensor collar around
their neck, which is hardly convenient for daily use [15]. The
main drawback of such methods is that they do not provide
useful nutrition data before the user eats. If people had more
information about their food, whether before eating it or even
before buying it, they might make more-informed decisions.
For this purpose, Manika et al. proposed a camera-based
method to determine the nutritional content of different foods
[16]. Their analysis, however, has to be done offline, which
also limits its usefulness in daily life. Some methods use
computer-vision algorithms and AR to attempt to combat
these problems. Waltner et al., for example, developed a system to provide food nutrition information in real-life scenarios [27]. But the food categories that it can recognize are
limited by traditional classification algorithms. The application
introduced by Stutz et al. aimed to assess nutrition information
[25]. Nevertheless, the system could not overlay nourishment
facts on targeted foods.
22 IEEE Consumer Electronics Magazine

MAy 2018

Concerning a related area, this is an exciting time for the
wearables market. Soon, many wearables intended for both
consumer and commercial applications will be made with
better-optimized hardware to address consumers' specific
needs [29]. Therefore, a wearable, scalable, and user-friendly
system for spotlighting food facts would be beneficial and
could be used in daily life, such as at a supermarket, to zero
consumers in on healthy foods.
In this article, we develop an AR-based head-mounted
system that enables the user to see food nutrition information
overlaid on items. It uses RIS (a form of content-based image
retrieval), text mining, and object tracking to superimpose
nutritional data on the corresponding food. The system makes
several contributions to the field of food nutrition monitoring.
▼ First, we went to a supermarket and tested a portable foodmonitoring system based on AR technology that retrieves
the nutrient facts and overlays them on the corresponding
foods with reasonable stability.
▼ Second, the system showed that food nutrition information
can be accessed and maintained in a scalable manner, specifically through the combination of doing an RIS and querying a remote database for nutritional data.
▼ Third, we evaluated the running time of the food-monitoring
system on a head-mounted display, specifically on Google
Glass. We also observed the runtime change of each frame
when the number of tracked food items increases.
▼ Finally, we experimentally evaluated the tradeoff between the
accuracy of the RIS and the speed of the tracking algorithm.

RELATED WORK
Using food-ingestion questionnaires is the most common method to understand an individual's dietary behavior. Traditional
methods for individual dietary assessments contain five general
categories: food records, 24-h dietary recalls, food frequency
questionnaires, diet histories, and food habit questionnaires
[18]. In a food-record survey, subjects report all foods and beverages consumed during a particular period. For 24-h dietary
recalls, a listing of foods and beverages consumed in the 24 h
prior to the recall interview is recorded. Researchers use foodfrequency questionnaires and diet histories to estimate food categories and consumption frequency over a defined time period,
such as a day, week, month, or year. Food-habit questionnaires
are designed to assess general or specific types of information,
such as food perceptions and beliefs, food likes and dislikes,
and so on. In practice, these five methods are usually combined
in some way. However, questionnaire-based methods are time
consuming and inaccurate and have poor compliance [12]. Participants may find questionnaires burdensome and must rely on
often untrustworthy memory to provide answers. Furthermore,
in [12], approximately 70% of users abandoned the long-term
questionnaires before completion.
Researchers have created more-convenient methods for
monitoring food intake with wearable and portable electronic
devices and systems. The types of sensors embedded in these
systems vary and include piezoelectric sensors, microphones,
and cameras. Ofei et al. [13] proposed a Dietary Intake

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - May 2018