Computational Intelligence - August 2017 - 14

TABLE 1 Mapping of likelihoods with accordance to World
Meteorological Organisation.
LIKELIHOOD OF
OCCURRENCE

LEXICALISATION

p 2 0.99

"EXTREMELY LIKELY"

0.90 # p # 0.99

"VERY LIKELY"

0.70 # p # 0.89

"LIKELY"

0.55 # p # 0.69

"PROBABLE - MORE LIKELY THAN NOT"

0.45 # p # 0.54

"EQUALLY LIKELY AS NOT"

0.30 # p # 0.44

"POSSIBLE - LESS LIKELY THAN NOT"

0.10 # p # 0.29

"UNLIKELY"

0.01 # p # 0.09

"VERY UNLIKELY"

p 1 0.01

"EXTREMELY UNLIKELY"

VI. Experimental Setup

TABLE 2 Description of collected demographic data.
VARIABLE

DESCRIPTION

AGE

USER COULD SPECIFY THEIR AGE AS
FREE TEXT

POSTCODE

ONLY THREE FIRST LETTERS WERE
REQUESTED

GENDER

MALE/FEMALE/OTHER

NATIVE SPEAKER OF
ENGLISH

YES/NO

FAMILIARITY WITH SCIENCE MODELS

HAVE YOU EVER BEEN TAUGHT OR LEARNT
ABOUT HOW -SCIENTISTS USE COMPUTERS
TO MODEL THE ENVIRONMENT?
POTENTIAL ANSWERS CAN BE: YES/NO/NOT
SURE

EXPERIENCE WITH
RISK

DO YOU OFTEN MAKE DECISIONS OR
JUDGEMENTS BASED ON RISK, CHANCE, OR
PROBABILITY? POTENTIAL ANSWERS CAN
BE: YES/NO/NOT SURE

EDUCATION

WHAT IS THE HIGHEST QUALIFICATION
YOU HAVE ACHIEVED SO FAR? POTENTIAL
ANSWERS CAN BE: DEGREE LEVEL OR
ABOVE, 2+ A-LEVELS OR EQUIVALENT,
APPRENTICESHIP, 5+ GCSES OR EQUIVALENT, 1-4 GCSES OR EQUIVALENT, OTHER
QUALIFICATIONS, NO QUALIFICATIONS.
WE ONLY FOUND SIGNIFICANT
DIFFERENCES BETWEEN BSC HOLDERS
AND NON-HOLDERS.

TABLE 3 Demographics of the adult cohort.
FEMALES
(OUT OF 197)

MALES
(OUT OF 241)

NATIVE SPEAKER OF ENGLISH

170 (86%)

199 (83%)

FAMILIARITY WITH SCIENCE
MODELS

79 (40%)

145 (60%)

EXPERIENCE WITH RISK

132 (67%)

185 (77%)

EDUCATION (BSC HOLDERS)

161 (82%)

205 (85%)

WMO-based system will generate the following forecast:
"Sunny intervals with rain being possible-less likely than not".
NATURAL: This system imitates forecasters and their natural way of reporting weather. The rules used in this system
have been derived by observing the way that experts (e.g. BBC
weather reporters) produce forecasts. For the previous example
(sunny intervals with 30% probability of rain), this system will
generate the following forecast: "Mainly dry with sunny spells".
This system is more natural than the WMO-based, in that the
probabilities are mapped to linguistic interpretation of weather
(e.g. "sunny spells") rather than the linguistic mapping of
uncertainty (e.g. "likely").

IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | AUGUST 2017

In order to investigate what helps people to better understand
and act upon uncertainty in information presentations, we initially use five conditions within the context of the Extended
Weather Game grouped in three categories:
Graphics only:
❏❏ This representation shows the users only the graphical representation of the weather forecasts. For this condition, we
used the graphs that scored best in terms of human comprehension in a study by Stephens et al. [36], [38].
Multi-modal representations:
❏❏ Graphics and NATURAL: This is a multi-modal representation consisting of graphics (as described in the previous
condition) and text produced by the NATURAL system.
❏❏ Graphics and WMO-based: This is also a multi-modal
representation consisting of graphics and text produced by
the WMO-based system.
NLG only:
❏❏ NATURAL only: This is a text-only representation as
generated by the NATURAL system.
❏❏ WMO-based system only: This is also a text-only representation as generated by the WMO-based system.
In the next sections, we treat the NLG systems under one category (NLG only) in order to be able to explore the effects that
NLG has on decision-making in general (also the results obtained
for the two NLG systems were not significantly different) compared to graphics only and to multi-modal representations.
Data

We recruited 442 unique adult players using social media (197
females, 241 males, 4 non-disclosed). We collected 450 game
instances (only a few people played the game twice). Further to
the Extended Weather Game results and the Berlin Literacy test
results, we also collected demographic data. A description of
the data is provided in Table 2. The full data set has been made
available online1.
The distributions of demographic features amongst our
adult participants are shown in Table 3. In the next section, we
describe the results with regard to game performance and

https://github.com/dimi123/WeatherGame

Table of Contents for the Digital Edition of Computational Intelligence - August 2017