IEEE Consumer Electronics Magazine - May 2018 - 77

say "select" to confirm a selection, e.g., instead of using an
air-tap gesture. Figures 3-5 are all captured with the voice
command "Hey Cortana, take a photo."

VISUALIZATION RESULTS
In this section, we use an example of visualizing Toronto voting data to illustrate our visualization results. As mentioned
previously, after the user gazes at the Voting Location 2014
option from the menu and selects it with an air-tap gesture,
all voting data are shown on the 3-D Toronto city model with
red and blue cuboids of different heights, as shown in Figure 5. Then the user can move his/her head to gaze at and airtap with a finger on a certain building on the city model to
obtain more detailed information. In Figure 5, the selected
building is transformed to orange and shows its detailed
information, including that the voter count is 2,413, address
is 65 Church, latitude is 43.65, and so forth. The user can
also move around the augmented model to view it from different angles and select other options in the menu (e.g., tennis courts) to visualize a new data set.

FIGURE 5. The visualized Toronto voting data, option menu, and
a specific building through interacting with HoloLens.

DISCUSSION
VISUALIZE CITY DATA WITH DIFFERENT TECHNOLOGIES
To compare our presented HoloLens AR method with the traditional computer monitor method in visualizing city data, we
conducted a user study to experimentally demonstrate the
advantages and limitations of each method. All users were
asked to complete a series of five tasks related to city data
visualization using HoloLens and a laptop as the main devices
to experience these two systems and then fill out a questionnaire. Because AR technology is able to display virtual contents upon a real environment, we also explored the capability
of combining with real haptic devices. These tasks were
▼ zoom in to watch a specific city region
▼ select a specific building
▼ change to another type of city data for visualization
▼ touch three designated buildings on the city model through
real haptic devices to feel force feedback for different city
data values (e.g., crime rate)
▼ move the city model to three places in one room.
With HoloLens, these tasks are completed through
1) walking close to the augmented 3-D city model to watch
the indicated region
2) gazing at the designed building
3) looking at the file menu and using a gesture to select the
desired city data type
4) touching three real haptic devices (Novint Falcon [19]) in
sequence, which are geographically overlapped with three
designated augmented buildings-then the user can feel different levels of preset force feedback to represent different
data values, as shown in Figure 6 (since HoloLens cannot
be directly connected with haptic devices, and this is a preliminary user study, each haptic device is simply connected
with an extra computer to preset matched force feedback
level through CHAI3D [20] C++ simulation framework)

FIGURE 6. A user touches an augmented building, which is geographically overlapped with a haptic device to provide force
feedback.

5) using gestures to move the city model to different places.
With a laptop, the corresponding actions are 1) moving
the mouse indicator to that region and rotating the mouse's
middle button to zoom in; 2) moving the mouse and clicking
the left button to choose; 3) using the mouse to select desired
data type; 4) watching designated buildings on screen in
sequence while touching corresponding haptic devices nearby assuming them as the controllers (as with the previous
step, each haptic device is connected with an extra computer
to preset matched force feedback level); and 5) picking up the
laptop and putting it at three places. The laptop adopted in
this user study is a Lenovo Y50, with a 15.6-in, 1,920 × 1,080
screen; Intel i7-4710HQ CPU; 16-gigabyte random-access
memory (RAM); Windows 10 OS; and a weight of 2.3 kg.
Each Novint Falcon haptic device is connected with an ordinary computer (e.g., with an Intel i7-920 processor, eightgigabyte RAM, and Windows 10 OS) to preset the corresponding
force feedback level.
After previous experiments, a questionnaire is provided for
users to rate each method in six aspects, including user interaction natural level, visual quality, combination with haptic devices, system response time, flexibility to move, and a general
rating. The rating scale for each question is from zero to five,
with steps of one. Based on the results of 12 users (six male and
six female), shown in Figure 7, HoloLens's average scores in
these aspects are 3.6, 4.4, 4.7, 2.9, 4.2, and 3.6, with standard
deviations of 1.1, 0.5, 0.5, 0.4, 0.7, and 0.5, whereas the laptop
MAy 2018

^

IEEE Consumer Electronics Magazine

77



Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - May 2018

IEEE Consumer Electronics Magazine - May 2018 - Cover1
IEEE Consumer Electronics Magazine - May 2018 - Cover2
IEEE Consumer Electronics Magazine - May 2018 - 1
IEEE Consumer Electronics Magazine - May 2018 - 2
IEEE Consumer Electronics Magazine - May 2018 - 3
IEEE Consumer Electronics Magazine - May 2018 - 4
IEEE Consumer Electronics Magazine - May 2018 - 5
IEEE Consumer Electronics Magazine - May 2018 - 6
IEEE Consumer Electronics Magazine - May 2018 - 7
IEEE Consumer Electronics Magazine - May 2018 - 8
IEEE Consumer Electronics Magazine - May 2018 - 9
IEEE Consumer Electronics Magazine - May 2018 - 10
IEEE Consumer Electronics Magazine - May 2018 - 11
IEEE Consumer Electronics Magazine - May 2018 - 12
IEEE Consumer Electronics Magazine - May 2018 - 13
IEEE Consumer Electronics Magazine - May 2018 - 14
IEEE Consumer Electronics Magazine - May 2018 - 15
IEEE Consumer Electronics Magazine - May 2018 - 16
IEEE Consumer Electronics Magazine - May 2018 - 17
IEEE Consumer Electronics Magazine - May 2018 - 18
IEEE Consumer Electronics Magazine - May 2018 - 19
IEEE Consumer Electronics Magazine - May 2018 - 20
IEEE Consumer Electronics Magazine - May 2018 - 21
IEEE Consumer Electronics Magazine - May 2018 - 22
IEEE Consumer Electronics Magazine - May 2018 - 23
IEEE Consumer Electronics Magazine - May 2018 - 24
IEEE Consumer Electronics Magazine - May 2018 - 25
IEEE Consumer Electronics Magazine - May 2018 - 26
IEEE Consumer Electronics Magazine - May 2018 - 27
IEEE Consumer Electronics Magazine - May 2018 - 28
IEEE Consumer Electronics Magazine - May 2018 - 29
IEEE Consumer Electronics Magazine - May 2018 - 30
IEEE Consumer Electronics Magazine - May 2018 - 31
IEEE Consumer Electronics Magazine - May 2018 - 32
IEEE Consumer Electronics Magazine - May 2018 - 33
IEEE Consumer Electronics Magazine - May 2018 - 34
IEEE Consumer Electronics Magazine - May 2018 - 35
IEEE Consumer Electronics Magazine - May 2018 - 36
IEEE Consumer Electronics Magazine - May 2018 - 37
IEEE Consumer Electronics Magazine - May 2018 - 38
IEEE Consumer Electronics Magazine - May 2018 - 39
IEEE Consumer Electronics Magazine - May 2018 - 40
IEEE Consumer Electronics Magazine - May 2018 - 41
IEEE Consumer Electronics Magazine - May 2018 - 42
IEEE Consumer Electronics Magazine - May 2018 - 43
IEEE Consumer Electronics Magazine - May 2018 - 44
IEEE Consumer Electronics Magazine - May 2018 - 45
IEEE Consumer Electronics Magazine - May 2018 - 46
IEEE Consumer Electronics Magazine - May 2018 - 47
IEEE Consumer Electronics Magazine - May 2018 - 48
IEEE Consumer Electronics Magazine - May 2018 - 49
IEEE Consumer Electronics Magazine - May 2018 - 50
IEEE Consumer Electronics Magazine - May 2018 - 51
IEEE Consumer Electronics Magazine - May 2018 - 52
IEEE Consumer Electronics Magazine - May 2018 - 53
IEEE Consumer Electronics Magazine - May 2018 - 54
IEEE Consumer Electronics Magazine - May 2018 - 55
IEEE Consumer Electronics Magazine - May 2018 - 56
IEEE Consumer Electronics Magazine - May 2018 - 57
IEEE Consumer Electronics Magazine - May 2018 - 58
IEEE Consumer Electronics Magazine - May 2018 - 59
IEEE Consumer Electronics Magazine - May 2018 - 60
IEEE Consumer Electronics Magazine - May 2018 - 61
IEEE Consumer Electronics Magazine - May 2018 - 62
IEEE Consumer Electronics Magazine - May 2018 - 63
IEEE Consumer Electronics Magazine - May 2018 - 64
IEEE Consumer Electronics Magazine - May 2018 - 65
IEEE Consumer Electronics Magazine - May 2018 - 66
IEEE Consumer Electronics Magazine - May 2018 - 67
IEEE Consumer Electronics Magazine - May 2018 - 68
IEEE Consumer Electronics Magazine - May 2018 - 69
IEEE Consumer Electronics Magazine - May 2018 - 70
IEEE Consumer Electronics Magazine - May 2018 - 71
IEEE Consumer Electronics Magazine - May 2018 - 72
IEEE Consumer Electronics Magazine - May 2018 - 73
IEEE Consumer Electronics Magazine - May 2018 - 74
IEEE Consumer Electronics Magazine - May 2018 - 75
IEEE Consumer Electronics Magazine - May 2018 - 76
IEEE Consumer Electronics Magazine - May 2018 - 77
IEEE Consumer Electronics Magazine - May 2018 - 78
IEEE Consumer Electronics Magazine - May 2018 - 79
IEEE Consumer Electronics Magazine - May 2018 - 80
IEEE Consumer Electronics Magazine - May 2018 - 81
IEEE Consumer Electronics Magazine - May 2018 - 82
IEEE Consumer Electronics Magazine - May 2018 - 83
IEEE Consumer Electronics Magazine - May 2018 - 84
IEEE Consumer Electronics Magazine - May 2018 - 85
IEEE Consumer Electronics Magazine - May 2018 - 86
IEEE Consumer Electronics Magazine - May 2018 - 87
IEEE Consumer Electronics Magazine - May 2018 - 88
IEEE Consumer Electronics Magazine - May 2018 - 89
IEEE Consumer Electronics Magazine - May 2018 - 90
IEEE Consumer Electronics Magazine - May 2018 - 91
IEEE Consumer Electronics Magazine - May 2018 - 92
IEEE Consumer Electronics Magazine - May 2018 - 93
IEEE Consumer Electronics Magazine - May 2018 - 94
IEEE Consumer Electronics Magazine - May 2018 - 95
IEEE Consumer Electronics Magazine - May 2018 - 96
IEEE Consumer Electronics Magazine - May 2018 - 97
IEEE Consumer Electronics Magazine - May 2018 - 98
IEEE Consumer Electronics Magazine - May 2018 - 99
IEEE Consumer Electronics Magazine - May 2018 - 100
IEEE Consumer Electronics Magazine - May 2018 - 101
IEEE Consumer Electronics Magazine - May 2018 - 102
IEEE Consumer Electronics Magazine - May 2018 - 103
IEEE Consumer Electronics Magazine - May 2018 - 104
IEEE Consumer Electronics Magazine - May 2018 - 105
IEEE Consumer Electronics Magazine - May 2018 - 106
IEEE Consumer Electronics Magazine - May 2018 - 107
IEEE Consumer Electronics Magazine - May 2018 - 108
IEEE Consumer Electronics Magazine - May 2018 - 109
IEEE Consumer Electronics Magazine - May 2018 - 110
IEEE Consumer Electronics Magazine - May 2018 - 111
IEEE Consumer Electronics Magazine - May 2018 - 112
IEEE Consumer Electronics Magazine - May 2018 - 113
IEEE Consumer Electronics Magazine - May 2018 - 114
IEEE Consumer Electronics Magazine - May 2018 - 115
IEEE Consumer Electronics Magazine - May 2018 - 116
IEEE Consumer Electronics Magazine - May 2018 - 117
IEEE Consumer Electronics Magazine - May 2018 - 118
IEEE Consumer Electronics Magazine - May 2018 - 119
IEEE Consumer Electronics Magazine - May 2018 - 120
IEEE Consumer Electronics Magazine - May 2018 - Cover3
IEEE Consumer Electronics Magazine - May 2018 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20240102
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20231112
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20230910
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20230708
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20230506
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20230304
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20230102
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20221112
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20220910
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20220708
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20220506
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20220304
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20220102
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20211112
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20210910
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20210708
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20210506
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_20210304
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_202010
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_202009
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_202007
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_202004
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_202003
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_202001
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201910
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201909
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201907
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201905
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201903
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201901
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201811
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201809
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201807
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201805
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_201803
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_july2017
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_april2017
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_january2017
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_october2016
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_july2016
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_april2016
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_january2016
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_october2015
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_july2015
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_april2015
https://www.nxtbook.com/nxtbooks/ieee/consumerelectronics_january2015
https://www.nxtbookmedia.com