IEEE Consumer Electronics Magazine - May 2018 - 87

map at the resort. During the ski run, the participants and the
researcher took pictures together, recorded tracks, and added a
few hazards that they encountered on the way. The actual
study session in front of the map took, on average, 15-30 min.
First, we briefed the one or two subjects (on two occasions we had a pair) on the goal of the study, requested their
consent, and then asked them to try the prototype, which
showed localized sample content previously entered into the
system by the researchers. In contrast to the laboratory study,
here we did not have any predefined scenarios, but rather
simply asked participants to decide where to go next, given
the current state of the content added to the system earlier
(see Figure 4).
In both investigations, we asked our subjects to complete a
poststudy questionnaire, in which they needed to indicate their
level of agreement on several statements regarding the usefulness of the SkiAR system, using a five-point Likert scale (see
Figure 5). Additionally, we administered an SUS questionnaire [8] to evaluate the usability of the system and a NASA
TLX workload test [7] to evaluate the mental, physical, and
temporal demands of the system. Finally, after both studies,
we conducted a semistructured interview. We recorded all
interviews using a voice recorder, then transcribed the recordings verbatim.
To analyze these data, we followed an iterative process,
going back and forth between them, the researchers' notes,
and the emerging structure of the empirical categories, which
was developed through repeated readings of the material. To
draw out the common factors in the system, we adopted a contextual design methodology and constructed an affinity wall
[13]. The goal of this part was to unveil the user experience
with the prototype, define ideas for new content and applications of the SkiAR system, and collect suggestions for the
design of technologies that could be used to support skiers

and snowboarders in group decision making on the slopes.
The qualitative evaluation helped us to better understand this
rich space descriptively and to shed some light on the design
challenges and opportunities of interactive systems for collaborative skiing.

RESULTS
The participants in both studies regarded the SkiAR system as generally useful to have during skiing or snowboarding. Figure 5(a)
shows that subjects in the field study generally gave higher scores
(higher perceived usefulness for the system in general, as well as
for each functionality-review, share, and add). In contrast, participants in the laboratory study especially appreciated the convenience that the system provides when watching the overlaid
information through the goggles and operating it through a wristmounted controller. We speculate that this may be because field
study participants were in a more realistic setting, facing an actual
decision-making activity.
Next, we were interested in evaluating the usability of the
head-mounted display setup, an approximation of the envisioned high-technology skiwear. The system achieved an average SUS score of 73.75 ^SD = 12.46h in the laboratory and
79.19 ^SD = 10.07h in the field study, which suggests that the
apparatus we developed is above average in usability [9].

4.00
4.33

System Is
Generally Useful

4.71

Convenient to Interact
with the System:

23.93
24.17
14.29
21.67
26.07
17.5
29.29
34.58

Mental Demand
Physical Demand

3.92

(i) Useful to Review
Information

4.07
4.25

(ii) Useful to Share
Information

3.93
4.25

(iii) Useful to Add
Information

3.86
4.17

1
Fully
Disagree

We built our prototype with a
vision of using ski goggles as
an output display to provide
additional information to skiers
and snowboarders.

Temporal Demand
Performance

23.93
29.17
20.71
25.42

Effort
Frustration
5
Fully
Agree

(a)

100

(b)
Laboratory Study

Field Study

FIGURE 5. (a) The perceived usefulness of the SkiAR system. (b) The NASA TLX average score on the content reviewing task, represented
on a logarithmic scale (a lower value is better). Laboratory study: N = 14. Field study: N = 12.

MAy 2018

IEEE Consumer Electronics Magazine

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - May 2018