Computational Intelligence - August 2017 - 14
TABLE 1 Mapping of likelihoods with accordance to World
Meteorological Organisation.
LIKELIHOOD OF
OCCURRENCE
LEXICALISATION
p 2 0.99
"EXTREMELY LIKELY"
0.90 # p # 0.99
"VERY LIKELY"
0.70 # p # 0.89
"LIKELY"
0.55 # p # 0.69
"PROBABLE - MORE LIKELY THAN NOT"
0.45 # p # 0.54
"EQUALLY LIKELY AS NOT"
0.30 # p # 0.44
"POSSIBLE - LESS LIKELY THAN NOT"
0.10 # p # 0.29
"UNLIKELY"
0.01 # p # 0.09
"VERY UNLIKELY"
p 1 0.01
"EXTREMELY UNLIKELY"
VI. Experimental Setup
TABLE 2 Description of collected demographic data.
VARIABLE
DESCRIPTION
AGE
USER COULD SPECIFY THEIR AGE AS
FREE TEXT
POSTCODE
ONLY THREE FIRST LETTERS WERE
REQUESTED
GENDER
MALE/FEMALE/OTHER
NATIVE SPEAKER OF
ENGLISH
YES/NO
FAMILIARITY WITH SCIENCE MODELS
HAVE YOU EVER BEEN TAUGHT OR LEARNT
ABOUT HOW -SCIENTISTS USE COMPUTERS
TO MODEL THE ENVIRONMENT?
POTENTIAL ANSWERS CAN BE: YES/NO/NOT
SURE
EXPERIENCE WITH
RISK
DO YOU OFTEN MAKE DECISIONS OR
JUDGEMENTS BASED ON RISK, CHANCE, OR
PROBABILITY? POTENTIAL ANSWERS CAN
BE: YES/NO/NOT SURE
EDUCATION
WHAT IS THE HIGHEST QUALIFICATION
YOU HAVE ACHIEVED SO FAR? POTENTIAL
ANSWERS CAN BE: DEGREE LEVEL OR
ABOVE, 2+ A-LEVELS OR EQUIVALENT,
APPRENTICESHIP, 5+ GCSES OR EQUIVALENT, 1-4 GCSES OR EQUIVALENT, OTHER
QUALIFICATIONS, NO QUALIFICATIONS.
WE ONLY FOUND SIGNIFICANT
DIFFERENCES BETWEEN BSC HOLDERS
AND NON-HOLDERS.
TABLE 3 Demographics of the adult cohort.
FEMALES
(OUT OF 197)
MALES
(OUT OF 241)
NATIVE SPEAKER OF ENGLISH
170 (86%)
199 (83%)
FAMILIARITY WITH SCIENCE
MODELS
79 (40%)
145 (60%)
EXPERIENCE WITH RISK
132 (67%)
185 (77%)
EDUCATION (BSC HOLDERS)
161 (82%)
205 (85%)
14
WMO-based system will generate the following forecast:
"Sunny intervals with rain being possible-less likely than not".
NATURAL: This system imitates forecasters and their natural way of reporting weather. The rules used in this system
have been derived by observing the way that experts (e.g. BBC
weather reporters) produce forecasts. For the previous example
(sunny intervals with 30% probability of rain), this system will
generate the following forecast: "Mainly dry with sunny spells".
This system is more natural than the WMO-based, in that the
probabilities are mapped to linguistic interpretation of weather
(e.g. "sunny spells") rather than the linguistic mapping of
uncertainty (e.g. "likely").
IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | AUGUST 2017
In order to investigate what helps people to better understand
and act upon uncertainty in information presentations, we initially use five conditions within the context of the Extended
Weather Game grouped in three categories:
Graphics only:
❏❏ This representation shows the users only the graphical representation of the weather forecasts. For this condition, we
used the graphs that scored best in terms of human comprehension in a study by Stephens et al. [36], [38].
Multi-modal representations:
❏❏ Graphics and NATURAL: This is a multi-modal representation consisting of graphics (as described in the previous
condition) and text produced by the NATURAL system.
❏❏ Graphics and WMO-based: This is also a multi-modal
representation consisting of graphics and text produced by
the WMO-based system.
NLG only:
❏❏ NATURAL only: This is a text-only representation as
generated by the NATURAL system.
❏❏ WMO-based system only: This is also a text-only representation as generated by the WMO-based system.
In the next sections, we treat the NLG systems under one category (NLG only) in order to be able to explore the effects that
NLG has on decision-making in general (also the results obtained
for the two NLG systems were not significantly different) compared to graphics only and to multi-modal representations.
Data
We recruited 442 unique adult players using social media (197
females, 241 males, 4 non-disclosed). We collected 450 game
instances (only a few people played the game twice). Further to
the Extended Weather Game results and the Berlin Literacy test
results, we also collected demographic data. A description of
the data is provided in Table 2. The full data set has been made
available online1.
The distributions of demographic features amongst our
adult participants are shown in Table 3. In the next section, we
describe the results with regard to game performance and
1
https://github.com/dimi123/WeatherGame
Table of Contents for the Digital Edition of Computational Intelligence - August 2017
Computational Intelligence - August 2017 - Cover1
Computational Intelligence - August 2017 - Cover2
Computational Intelligence - August 2017 - 1
Computational Intelligence - August 2017 - 2
Computational Intelligence - August 2017 - 3
Computational Intelligence - August 2017 - 4
Computational Intelligence - August 2017 - 5
Computational Intelligence - August 2017 - 6
Computational Intelligence - August 2017 - 7
Computational Intelligence - August 2017 - 8
Computational Intelligence - August 2017 - 9
Computational Intelligence - August 2017 - 10
Computational Intelligence - August 2017 - 11
Computational Intelligence - August 2017 - 12
Computational Intelligence - August 2017 - 13
Computational Intelligence - August 2017 - 14
Computational Intelligence - August 2017 - 15
Computational Intelligence - August 2017 - 16
Computational Intelligence - August 2017 - 17
Computational Intelligence - August 2017 - 18
Computational Intelligence - August 2017 - 19
Computational Intelligence - August 2017 - 20
Computational Intelligence - August 2017 - 21
Computational Intelligence - August 2017 - 22
Computational Intelligence - August 2017 - 23
Computational Intelligence - August 2017 - 24
Computational Intelligence - August 2017 - 25
Computational Intelligence - August 2017 - 26
Computational Intelligence - August 2017 - 27
Computational Intelligence - August 2017 - 28
Computational Intelligence - August 2017 - 29
Computational Intelligence - August 2017 - 30
Computational Intelligence - August 2017 - 31
Computational Intelligence - August 2017 - 32
Computational Intelligence - August 2017 - 33
Computational Intelligence - August 2017 - 34
Computational Intelligence - August 2017 - 35
Computational Intelligence - August 2017 - 36
Computational Intelligence - August 2017 - 37
Computational Intelligence - August 2017 - 38
Computational Intelligence - August 2017 - 39
Computational Intelligence - August 2017 - 40
Computational Intelligence - August 2017 - 41
Computational Intelligence - August 2017 - 42
Computational Intelligence - August 2017 - 43
Computational Intelligence - August 2017 - 44
Computational Intelligence - August 2017 - 45
Computational Intelligence - August 2017 - 46
Computational Intelligence - August 2017 - 47
Computational Intelligence - August 2017 - 48
Computational Intelligence - August 2017 - 49
Computational Intelligence - August 2017 - 50
Computational Intelligence - August 2017 - 51
Computational Intelligence - August 2017 - 52
Computational Intelligence - August 2017 - 53
Computational Intelligence - August 2017 - 54
Computational Intelligence - August 2017 - 55
Computational Intelligence - August 2017 - 56
Computational Intelligence - August 2017 - 57
Computational Intelligence - August 2017 - 58
Computational Intelligence - August 2017 - 59
Computational Intelligence - August 2017 - 60
Computational Intelligence - August 2017 - 61
Computational Intelligence - August 2017 - 62
Computational Intelligence - August 2017 - 63
Computational Intelligence - August 2017 - 64
Computational Intelligence - August 2017 - 65
Computational Intelligence - August 2017 - 66
Computational Intelligence - August 2017 - 67
Computational Intelligence - August 2017 - 68
Computational Intelligence - August 2017 - 69
Computational Intelligence - August 2017 - 70
Computational Intelligence - August 2017 - 71
Computational Intelligence - August 2017 - 72
Computational Intelligence - August 2017 - 73
Computational Intelligence - August 2017 - 74
Computational Intelligence - August 2017 - 75
Computational Intelligence - August 2017 - 76
Computational Intelligence - August 2017 - Cover3
Computational Intelligence - August 2017 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202311
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202308
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202305
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202302
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202211
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202208
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202205
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202202
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202111
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202108
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202105
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202102
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202011
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202008
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202005
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202002
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201911
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201908
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201905
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201902
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201811
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201808
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201805
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201802
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter12
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall12
https://www.nxtbookmedia.com