Computational Intelligence - May 2013 - 57
model of the world or the predefined rules on which action to
choose. Hence, we adopt a modeless reinforcement learning
mechanism, Q-learning [33], for the agent. The goal of
Q-learning is to estimate the Q(s, a) values, expected rewards for
executing action a in state s. Each Q(s, a) is updated based on
Q ^s, ah = ^1 - ~h $ Q ^s, ah + ~ $ ` r + cV ^slh j ,
(15)
V ^slh = max a ! Action Q ^sl, ah
(16)
where
is the best reward the agent can expect from the new state sl .
Here, a is an action and Action is the set of actions available for
the agent. r is the reinforcement value. c is the discount factor
that defines how much expected future rewards affect decision
now. ~ is the learning rate that controls how much weight is
given to the reward just experienced.
4) the Roles of Curiosity-Related Emotions
Emotions perform vital functions in human decision-making.
Psychological studies show that humans voluntarily seek novel
things due to the pleasure of satiating curiosity [17]. Artificial
Intelligence research frequently assumes that the human decision-making process consists of maximizing positive emotions
and minimizing negative emotions [29], [34], [30]. Based on
these observations, curiosity-related emotions are employed as
reinforcement functions to guide the virtual peer learner's learning of behavior strategies. The emotion curiosity gives positive
reinforcement, while both boredom and anxiety give negative
reinforcement.
Also, in humans, emotion can influence action strengths
[31]. For example, a student who is interested in a subject will
concentrate more and achieve higher learning efficiency than
one who is bored with the same subject. Hence, the second
role of curiosity-related emotions is to influence actions. Here,
we consider their influence on action study. The virtual peer
learner's learning efficiency D can be influenced by emotions
as follows:
D = l + F ^emotionh ,
trated in Fig. 4, where Fig. 4(a) shows the landscape of VS,
designed based on 19th century Singapore. Fig. 4(b) shows the
ant hole trigger where students can shrink their avatars to go
inside the tree environment.
For the proposed curious peer learner, we follow the methodology of Learner-Centered Design [35]. It advocates that
technology applications must focus on the needs, skills, and interests of the learner. Before developing the curious peer learner,
we had described the functionalities of a curious peer learner
and illustrated possible interaction scenarios. Some examples are
shown in Fig. 5. It can be observed from Fig. 5(a) that the curious peer learner (embodied by a butterfly) is posing questions to
a student in order to stimulate his/her thinking on the learning
content. Fig. 5(b) demonstrates that the curious peer learner is
directing the student's attention to the potentially interesting
learning content.
We conducted a pre-study among several students who
have played VS. Responses from questionnaires and interviews have shown that students are interested in having a
(a)
(17)
where F is a mapping from the emotion to its influence on the
agent's learning efficiency. Here, F(curiosity) returns a positive
value, while F(boredom) and F(anxiety) both return negative values. D is always capped between 0 and 1.
VI. Experiment
In this section, we present the experimental details and the
experimental results obtained.
A. Virtual Singapura
Virtual Singapura (VS) is a virtual learning environment
designed for lower secondary school students to learn the plant
transport systems. The virtual world environment in VS is illus-
(b)
Figure 4 The virtual environment in VS. (a) The VS landscape
designed based on the 19th century Singapore. (b) The ant hole
trigger where students can shrink their avatars and go inside a tree.
may 2013 | IEEE ComputatIonal IntEllIgEnCE magazInE
57
Table of Contents for the Digital Edition of Computational Intelligence - May 2013
Computational Intelligence - May 2013 - Cover1
Computational Intelligence - May 2013 - Cover2
Computational Intelligence - May 2013 - 1
Computational Intelligence - May 2013 - 2
Computational Intelligence - May 2013 - 3
Computational Intelligence - May 2013 - 4
Computational Intelligence - May 2013 - 5
Computational Intelligence - May 2013 - 6
Computational Intelligence - May 2013 - 7
Computational Intelligence - May 2013 - 8
Computational Intelligence - May 2013 - 9
Computational Intelligence - May 2013 - 10
Computational Intelligence - May 2013 - 11
Computational Intelligence - May 2013 - 12
Computational Intelligence - May 2013 - 13
Computational Intelligence - May 2013 - 14
Computational Intelligence - May 2013 - 15
Computational Intelligence - May 2013 - 16
Computational Intelligence - May 2013 - 17
Computational Intelligence - May 2013 - 18
Computational Intelligence - May 2013 - 19
Computational Intelligence - May 2013 - 20
Computational Intelligence - May 2013 - 21
Computational Intelligence - May 2013 - 22
Computational Intelligence - May 2013 - 23
Computational Intelligence - May 2013 - 24
Computational Intelligence - May 2013 - 25
Computational Intelligence - May 2013 - 26
Computational Intelligence - May 2013 - 27
Computational Intelligence - May 2013 - 28
Computational Intelligence - May 2013 - 29
Computational Intelligence - May 2013 - 30
Computational Intelligence - May 2013 - 31
Computational Intelligence - May 2013 - 32
Computational Intelligence - May 2013 - 33
Computational Intelligence - May 2013 - 34
Computational Intelligence - May 2013 - 35
Computational Intelligence - May 2013 - 36
Computational Intelligence - May 2013 - 37
Computational Intelligence - May 2013 - 38
Computational Intelligence - May 2013 - 39
Computational Intelligence - May 2013 - 40
Computational Intelligence - May 2013 - 41
Computational Intelligence - May 2013 - 42
Computational Intelligence - May 2013 - 43
Computational Intelligence - May 2013 - 44
Computational Intelligence - May 2013 - 45
Computational Intelligence - May 2013 - 46
Computational Intelligence - May 2013 - 47
Computational Intelligence - May 2013 - 48
Computational Intelligence - May 2013 - 49
Computational Intelligence - May 2013 - 50
Computational Intelligence - May 2013 - 51
Computational Intelligence - May 2013 - 52
Computational Intelligence - May 2013 - 53
Computational Intelligence - May 2013 - 54
Computational Intelligence - May 2013 - 55
Computational Intelligence - May 2013 - 56
Computational Intelligence - May 2013 - 57
Computational Intelligence - May 2013 - 58
Computational Intelligence - May 2013 - 59
Computational Intelligence - May 2013 - 60
Computational Intelligence - May 2013 - 61
Computational Intelligence - May 2013 - 62
Computational Intelligence - May 2013 - 63
Computational Intelligence - May 2013 - 64
Computational Intelligence - May 2013 - 65
Computational Intelligence - May 2013 - 66
Computational Intelligence - May 2013 - 67
Computational Intelligence - May 2013 - 68
Computational Intelligence - May 2013 - 69
Computational Intelligence - May 2013 - 70
Computational Intelligence - May 2013 - 71
Computational Intelligence - May 2013 - 72
Computational Intelligence - May 2013 - 73
Computational Intelligence - May 2013 - 74
Computational Intelligence - May 2013 - 75
Computational Intelligence - May 2013 - 76
Computational Intelligence - May 2013 - 77
Computational Intelligence - May 2013 - 78
Computational Intelligence - May 2013 - 79
Computational Intelligence - May 2013 - 80
Computational Intelligence - May 2013 - Cover3
Computational Intelligence - May 2013 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202311
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202308
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202305
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202302
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202211
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202208
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202205
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202202
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202111
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202108
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202105
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202102
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202011
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202008
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202005
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202002
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201911
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201908
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201905
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201902
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201811
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201808
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201805
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201802
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter12
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall12
https://www.nxtbookmedia.com