IEEE Computational Intelligence Magazine - November 2021 - 67

and have full combat capabilities, including fighting enemies
and utilizing available resources such as weapons and medical
kits. All agents need to focus on killing opponents and using
the available resources to die as few times as possible. For the
fairness of comparison, all simulation experiments were conducted
on the same map, and all agents had the same resources.
In addition, hunters and memetic agents had no prior
knowledge of each other's position and could only choose
among the predefined actions. Memetic agents and hunters
were assigned into two separate teams. Each team had N combat
agents. Multiple battles were carried out between memetic
agents and hunters.
1) Comparison with State-of-the-Art Methods
The aim of the first set of experiments was to investigate the
performance of the proposed method, labeled as d-MeMBi,
compared to the existing methods, including Baseline, MeMAS,
and eTL, in a 4v4 Deathmatch game scenario. Note that a
detailed description of these experiments can be found in Section
V-A. In this study, d-MeMBi integrates both the proposed
environmental attention control (Section III) and the bidirectional
imitation strategy (Section IV).
Figure 16 depicts the learning trends of agents in KD
under Baseline, MeMAS, eTL, and the proposed d-MeMBi. In
general, d-MeMBi obtained the superior fighting performance
in terms of KD (i.e., 1.629) throughout the learning process,
which is slightly better than eTL (i.e., 1.583) and significantly
higher than Baseline (i.e., 1.194) and MeMAS (i.e., 1.327). Note
that an average KD of 1.6 at the end of the learning process
indicates that each agent could kill 1.6 enemies before dying.
The results indicate that agents under d-MeMBi and eTL had
significantly better combat performance in killing their opponents
while avoiding being killed, thus verifying the effectiveness
of the proposed method for completing missions in the
complex battle environment.
To evaluate the efficacy of the proposed d-MeMBi more
comprehensively, the experimental results of memotype number
generated as learning progressed are presented in Figure 17,
and the experimental results of communication cost regarding
imitation number after the learning process are presented in
Figure 18.
The memotype number (MN) essentially measures the knowledge
generalization capability of memetic agents. Under a similar
level of combat performance (i.e., similar KD), the smaller
the MN, the higher the knowledge generalization level of an
agent. According to the result in Figure 17, the proposed
d-MeMBi had the lowest number of memotypes generated while
completing the combat mission (i.e., 102), yet still obtained the
highest combat performance (i.e.,
KD .1 629=
). This result
demonstrates the superiority of d-MeMBi over eTL, which
achieved competitive KD performance (i.e., 1.583) but yielded
more memotypes (i.e., 110) after the learning process. Thus, d-MeMBi
exhibited a greater knowledge generalization capability.
Furthermore, the efficiency of the social interaction driven
by imitation pertaining to the communication cost was
considered. Figure 18 depicts the imitation numbers of
MeMAS, eTL, and the proposed d-MeMBi. It is obvious that d
-MeMBi incurred the lowest cost of imitation at
IN ,,10 000.
which was much less than the imitation number of MeMAS
1.6
1.2
0.8
Baseline
MeMAS
eTL
δ-MeMBi
0.4
0 246
Learning Steps
×104
FIGURE 16 Kill Death ratio of agents under Baseline, MeMAS,
eTL, and d-MeMBi in completing learning missions in Unreal
Tournament 2004.
120
100
80
Baseline
MeMAS
eTL
δ-MeMBi
60
0246
Learning Steps
×104
FIGURE 17 Memotype Number of agents under Baseline, MeMAS,
eTL, and d-MeMBi while completing learning missions in Unreal
Tournament 2004.
1.5
2
0.5
1
MeMAS eTL δ-MeMBi
FIGURE 18 Imitation Number of agents under MeMAS, eTL, and
d-MeMBi while completing learning missions in Unreal Tournament
2004.
×105
NOVEMBER 2021 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 67
Memotype Number
Kill Death Ratio
Imitation Number
IEEE Computational Intelligence Magazine - November 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - November 2021
