IEEE Consumer Electronics Magazine - May 2018 - 92

If the adoption curve for VR
and AR technologies is similar to
smartphones, we may be using
these technologies daily in the
next seven to ten years.

Despite the fact that AR poses major perceptual challenges, data representations in AR are rarely perceptually validated, with the exception of the detailed studies on legibility by
Gabbard et al. [10]. Real-world things may interfere with the
correct decoding of AR visualizations and have negative
effects on scene coherence [16]. Therefore, AR interfaces
should take into consideration not only what information to
show users, but also when and how to present it.

A CONNECTED COFFEE MACHINE
Virtual-reality (VR) and AR technology hold the potential
to replace traditional displays in many contexts. As soon as VR
and AR devices are mature enough to provide a flawless and
enjoyable experience at a reasonable price, they will become a
profoundly disruptive technology. If the adoption curve for VR
and AR technologies is similar to smartphones, we may be
using these technologies daily in the next seven to ten years
[24]. It therefore becomes crucial to consider how these technologies can help turn the often-challenging complexity of
modern machinery into an appealing experience. Applications
to visualize complex machinery in AR have taken various
approaches, and different displays are used to realize them.
The majority of AR applications (e.g., marketing, commercial, professional, educational, and entertainment) focus
on visual stimuli. These applications typically use AR displays, which can be classified as head-mounted (e.g., glasses
and helmets), handheld (e.g., smartphones and tablets), and
spatial (e.g., video projectors and holograms) displays, each
type offering advantages for specific use cases. Handheld displays are now mainstream, a ubiquitous AR display owing to
the broad adoption of smartphones. Head-mounted displays
have supporters in professional areas that require hands-free
interaction with real objects. Wearable devices like smart
glasses have the power to bring AR into everyday life [28].
We implemented our prototype in a minimalistic version of
the virtual showcase [31].
Spatial AR was effectively used for IoT-like applications,
as in an augmented kitchen where appliances communicate to
people (e.g., listing the contents of the refrigerator) and support users without interfering with the task itself (e.g., projecting a recipe or pointing to a cabinet) [18], [19]. Notably,
research on AR visualization has concentrated on metaphors
to present comprehensive instructions using the technique
[19]. Prominent metaphors have adopted known artistic methods of visual communication and concentrated on generating
multiperspective explosion diagrams [19], dynamic label
placement techniques [19], and ghosted views in AR [19].
With respect to information from sensors, conventional
visualizations have been used in AR to visualize multivariate,
real-time sensory data [2]. An important issue for comprehensive, coherent AR methods is the need to be adaptive to
changes in viewpoint, illumination, and colors [19]. Close to
our concept, Pokric et al. deployed an AR application on
handheld devices (Android and iOS), featuring a virtual 3-D
avatar on a marker. The measurements obtained from an IoT
environmental sensor affect the avatar aspect [30].
92 IEEE Consumer Electronics Magazine

MAy 2018

A coffee machine is a domestic device that people use in
their daily lives. Although its operation is simple, several
conditions must be observed both for a quality cup of coffee
and the health of the device itself. People understand the
operative factors of a coffee machine but often miss the maintenance aspects that can harm both the machine's health and
the coffee's quality. Several coffee machine models belong to
the new generation of IoT devices, but there are no opensource application programming interfaces to access their
sensor data.
Therefore, we constructed a customized IoT coffee
machine that matched our requirements, following an
approach of adapting nonconnected devices to create IoT
gadgets out of ordinary consumer goods [27]. Both hardware
and software solutions are available on the market for such
do-it-yourself projects [25].

HARDWARE
We extended a basic coffee machine with sensors of three
types: an Ultra-Sonic Sensor HC-SR04 in the water tank to
measure the water level, two force-sensitive resistors (one at
the top of the coffee capsule drawer to detect the presence or
absence of the capsule and the other over the on-off button to
record the button press), and a Temperature Sensor LM35
inside the machine to measure the water temperature.
A microcontroller collects the data from the sensors and
uses an Ethernet module to transfer them to an MQTT server
(lightweight message protocol). For convenience, we built a
hardware interface to set the desired status of the machine.
This feature allowed us to manipulate the sensor data (e.g., to
increase the temperature) and simulate critical errors merely
by pressing a button.
Several operational aspects of this basic prototype are
transferrable to other machines in more complex processes:
raw materials are used to produce a product (e.g., coffee capsules to produce coffee), fluids must be maintained within
acceptable ranges (e.g., the amount of water for the coffee),
waste materials have to be removed (e.g., used coffee capsules), the temperature of the machine must fluctuate within
an acceptable range, and maintenance is required to ensure the quality of the product and the health of the device
(e.g., decalcification).

IMMERSIVE DISPLAY
Devices hosting the BioAR application pick up and display
the sensor data. We built the application in a cross-platform

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - May 2018