IEEE Computational Intelligence Magazine - November 2021 - 59

between the primary analysis and the advanced meaning
analysis. For example, the Broadbent Filter Model merely
allows information from one channel to pass, whereas the
Treisman Attenuation Model allows information from all
channels to pass at different intensities [27].
Inspired by the study of human attention in psychology, this
study introduces an attention control model into an agent's
learning process when information is received from different
sensors. In particular, the intensity attenuation parameter d is
proposed to adjust the intensity of environmental information
from different sensory units so that the agent can focus on the
important information, which is expected to be remembered
more clearly and analyzed more carefully.
Specifically, the intensity of environmental states captured
by different sensors is adjusted to change the degree to which
the information affects the current task. In MeMAS, memes
are coined as the memory items in the agent's mind universe
for instructing behaviors in the current state, providing the
previously learned reward estimation. Meme expression, especially
meme activation, is the core process used to make a
choice and activate the most appropriate meme for solving
the given sensory input. For this core process, a new meme
activation strategy was proposed with the intensity attenuation
parameter d as follows.
L
Tj =
/ xwj
i =1
ad
d
i
L
c1 / wj
i =1
+
i
where L denotes the number of state channels captured by
different sensors, id denotes the intensity value of signals from
the ith sensory channel, x ,ci1
ith sensory channel, wj
denotes the state input of the
implies the
ci1
j
weights of the jth memotype (or neuron)
in the memory pool, and w ,
w .j
c1
Notably, d ! [, ]01 and
Ri =1 id = 1 . Generally, a higher id indiL
cates
that an agent pays more attention
to the state information from the ith
sensory channel.
Importantly, the agents are expected
to identify the most appropriate
intensity attenuation parameters automatically
for a specific problem domain.
A natural idea is to estimate the intensity
or clarity of the useful memory
(memotypes) on the information captured
from different sensory channels.
Therefore, a method is proposed to calculate
the intensity attenuation parameter
d by analyzing the sensory state
components of the useful memotypes
collected during the learning process.
Activated
Memes
···
|MIS|{I1,I2...IL}
Meme Expression
and Activation
···
ol
Meme Pool
δ Adaptation {I1
δ1
δ2
···
δL
Memetic Agent
Environment
Attention Controller
FIGURE 3 Illustration of meme expression with environment attention control.
Median
|MIS|
Median, I2
{I1}{I2}{IL}
{I1}{I2}{IL}
···
:::
Median ... IL
Median}
denotes
the ith state component of the weight
vector
MIS
1{I1,I2...IL}
2{I1,I2...IL}
·
·
1
MIS1
MIS2 MISL
Sorted MIS
{I1}{I2}{IL}
:::
1
ci,
ci,1 /
1
ci,
(9)
The method is elaborated in Algorithm 2 and Figure 3. In
particular, for a given memotype j, the memory intensity (clarity
of memory) ,,
Ii
i 12 Lf=
channel is calculated below.
I
i
=
ci1
j
xw
,
ci,1 /
1
ci
x
which implies the distance between the ith state component of
the weight vector w ,
and the ith component of the state
Algorithm 2 Adjustment of Attention Controller.
1 Initialization:
di 11 Lf== ; MIS = {}.
/,
Li
2 while stop conditions are not satisfied do
3
Initialize MIO = {}.
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19 End
while current state s is not terminal do
Perform memotype expression.
Calculate =
end
if agent succeeds in this trial then
MISMIS MIO,
=
.
end
/*After certain learning trials*/
Sort MISI II iL
i ii i
12 3
Identify IMedian
i
==12
{, ,...},
of MI i ,,..., .
Conduct Normaliz (),, ... .
Update ,, ,, .iLdi 12 f=
MIS = {}.
eI iL
Si = 1 L
=
Median 12
i
2 Equation 11.
,... .
II { ... }.IL
1
Put the element I into MIO.
Continue meme internal evolution.
2 Equation 10.
1
ci,
j
on the information of the ith state
(10)
NOVEMBER 2021 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 59
IEEE Computational Intelligence Magazine - November 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - November 2021
