IEEE Computational Intelligence Magazine - November 2021 - 57

Input vectors: Let S (, ,, ) denote the state
input vector and sn
= fss sn12
A aa am12= f (, ,, ) denote the action vector, where am
R 1)
(,
rr
represent the nth sensory input. Let
indicates
the mth possible action. Let =- denote the reward
vector, where r [, ]01!
denote the Fck
1
. All input values of FALCON are
assumed to be bounded between 0 and 1. The normalization
is necessary if the original values are not in the appropriate range.
Activity vectors: Let xck
activity vector for
k ,, ,12 3 where xx
=
F .2
12 and xc3
cc
,,
Weight vectors: Let wj
1
=
ck
correspond to the input state
S, action set A, and reward R, respectively. Let y denote the output
of the activity vector
associated with the jth neuron in F2
activity vector Fck
represent the weight vector
for learning the input
for k ,, .12 3 F2 has only one uncommitted
neuron (labeled as the green node in Figure 2) when
learning starts. When an uncommitted neuron chooses to
learn an association, it will become a committed state
(labeled as red nodes in Figure 2), and a new uncommitted
node will be initialized.
In the design of MeMAS, memes can be categorized into
memotypes and sociotypes. Internally, a memotype is defined as a
neuron stored in the cognitive layer F2
that encodes a semantic
rule mapping of the relationship between the agent's states S
and behavioral actions A, providing the environmental reward
estimation R. Externally, a sociotype is a behavior or action
expressed by agents that can be discovered and imitated by
other agents.
1) Meme Expression
Memotype activation: The input activity vector is propagated
from the bottom to the top and calculates the activation
value of each meme (cognitive neuron) in the F2
cally, given the activity vector x ,ck
field. Specifimeme
activation calculates the
similarity between the activity and weight vectors of each
meme j in
F ,2 defined as T .j
Tj = / ck xwj
a + wj
3
c
k=1
where the fuzzy AND operation / is defined as ()
is defined as p / R Both the
and the contribution parameter
pq i
mi (, ),ii and the norm $
ck 02
n pq
choice parameter a
ck [, ]01!
c
are predefined.
Memotype competition: Memotype competition identifies
the highest activation value of the encoded memotype in the F2
layer. After memotype activation, the meme with the highest activation
value in F2
F .2
The index of the winner J is calculated in Equation 2:
{:
TT Fjforall nodeJj 2
= max
}.
1 ,
(2)
State (xc1)
When a choice is made on meme J, yJ = and the activity
vectors of the other memes are set to 0.
Sociotype readout: After a winner neuron J (or memotype)
in the category field F2
is selected in memotype competition,
Action (xc2)
Input Layer
FIGURE 2 The FALCON network structure with three input sensory
fields.
Reward (xc3)
wc1
is identified. Let y be the activity vector of
Sensory
Field
Motor
Field
Feedback
Field
wc2
F2 Layer
wc3
p .
ii
/ /
/
ck
ck
ck
ck
(1)
2) Meme Assimilation
Memotype matching: Before using the selected winner meme
J for learning, memotype matching is introduced to determine
whether the weight vector of meme J is sufficiently similar to
their respective activity vector. Only when the matching function
mJ
ck of each channel k satisfies the vigilance parameters does
resonance occur.
ck
m =
J
ck
xwj
ck
/
x
where the vigilance parameter ck
=
t is predefined, and t
ck [, ]01!
for k ,, .12 3 If any matching value does not meet the vigilance
parameters, a mismatch reset will occur, the activation
value of TJ
will be set to 0, and another committed or uncommitted
meme will be selected for matching again until resonance
is achieved.
Memotype learning: If memotype matching is successful, the
weight vector (of memotype) wJ
ck
ww xw ()
J
ck
()=-bb /
()
newold
ck
J
ck
+
ck
ck
J
ck
old
in each channel k is then
updated through memotype learning below.
() ()1
ck [, ]01!
where k ,,= 12 3 and b
b is usually set to 1. For a
ck
(5)
represents the learning rate.
The basic principle of memotype learning is to learn through
the common attribute values of input and weight vectors.
For an uncommitted meme,
committed meme, b is set to a high value for fast learning
ck
in simple situations, or a small value for slow learning in
noisy environments. When the uncommitted meme is selected
for learning, a new uncommitted neuron is added into
the F2
layer. Therefore, the memetic agent can dynamically
expand its network structure.
ck
$ t
ck
(4)
Finally, the resulting Fck
1
activity vectors are the fuzzy AND of
their original values and weight vectors
w .J
ck
the winner neuron will perform a readout of its action into the
input fields Fck
1 by Equation 3.
ck newold
xx .w
()
=
ck()
/
J
ck
(3)
NOVEMBER 2021 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 57
IEEE Computational Intelligence Magazine - November 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - November 2021
