IEEE Consumer Electronics Magazine - May 2018 - 93

game engine. It runs an MQTT client that subscribes to sensor data streams from the outfitted coffee machine and presents the corresponding visual representation.
The interaction between man and machine is mediated by
the nature metaphor. We deployed two applications: an AR
application running on a smartphone and a holographic application working on a simple display. As the user operates the
machine and the sensors pick up different conditions, the AR
display reflects the status changes.
The AR application uses an AR SDK module to register the
visualization with a marker (see Figure 1). Other researchers
have previously used AR to display multivariate, real-time sensory data using conventional visualizations [2]. BioAR uses a
marker next to the machine; it is detected and tracked using the
AR SDK internally, and a 3-D tree representing the machine
status is displayed. The AR application presents a plausible
solution to visualize the metaphor, bringing into reality a sensing representation of the machine. The disadvantage of AR is
that a device is required to actually mediate the view, and putting the apparatus down disrupts the metaphor.
We constructed the holographic application to counter this
disadvantage. Figure 2 depicts the prototype structure. It consists
of a flat-panel monitor connected to a personal computer and
a flexiglass pyramid that reflects the four-side rendering of
the 3-D object, creating the illusion that the projected object
is floating in midair. The desktop application creates a multiple-surface projection of a 3-D tree model, using four cameras with a 90° interval to capture four views of the model. An
arbitrary number of users can experience the holographic
application, but it requires a larger infrastructure sitting next
to the machine. An advantage is that the experience is available even without pulling out one's phone.

As the user operates the machine
and the sensors pick up different
conditions, the AR display reflects
the status changes.
ator is expected to infer the general state of a machine by
assessing the tree appearance. To get more details, a user can
visually examine a specific property of the tree.

VISUAL ENCODING
The tree metaphor we designed to encode the machine status
allows for three main elements: water level, temperature, and
elapsed time before maintenance.
1) The water level of the tank is represented by the thickness
of the tree's greenery. Figure 3 depicts the three considered
levels: low-density foliage (the tank is less than one-third
full), medium-density leaves (the water level is greater than
one-third but less than two-thirds of the way up), and dense
foliage (the tank is more than two-thirds full).

BioAR: A NATURE-INSPIRED VISUAL ENCODING
We used a tool kit to create several 3-D models of the trees,
which represent the different states of the coffee machine.
These 3-D models are the basis of the BioAR tree metaphor.

TREE METAPHOR
In our nature-inspired metaphor, trees represent single
machines. The machines' sensory data are encoded into the
various visual features of the BioAR tree metaphor. The oper-

(a)

FIGURE 1. The BioAR, seen through a handheld AR device (a 5.5-in
mobile phone). The BioAR nature metaphor represents the sensor
data that define the status of the coffee machine in the features
of a virtual tree.

(b)

(c)

FIGURE 2. A BioAR holographic display. (a) The sensor-outfitted coffee machine streams sensor data that are encoded in a holographic
tree. (b) The structure of the holographic display consisting of a 17-in flat monitor and a flexiglass pyramid. (c) A detail of the four-side
rendering of the 3-D object.

MAy 2018

IEEE Consumer Electronics Magazine

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - May 2018