IEEE Computational Intelligence Magazine - November 2021 - 65

significantly higher than all other methods. Again, this result
shows the efficiency of the proposed bidirectional imitation strategy
for improving the learning performance of agents.
In addition, more comprehensive experiments were conducted
to testify the effectiveness of our proposed bidirectional
imitation strategy in an ablative manner. The performance of
both Forward and Backward imitation was evaluated on completing
the MNT tasks. The complete results of SR, MN, IN,
SN, SI, and RT for these methods are summarized in Table II.
Figures 11 to 13 show their corresponding learning curves
pertaining to SR, MN, and IN, respectively.
According to the results in Figure 11 and Table II, both
Bi-MeMAS and Backward Imitation had significantly better performance
than Forward Imitation, which reported the lowest
SR of 63.8% while incurring the highest imitation cost at an
IN of 9.47E+04. Furthermore, Bi-MeMAS and Backward Imitation
obtained competitive performance in terms of SRs
across the learning process. Nevertheless, agents under BiMeMAS
spent one third of the imitation cost compared to
Backward Imitation (IN = 1.34E+04 versus IN = 3.89E+04).
Note that these two methods generated a similar number of
memotypes after the learning process, and Bi-MeMAS had significantly
higher SN and SI than Backward Imitation. In summary,
these results clearly verify the superior efficacy of the
proposed bidirectional imitation strategy for enhancing the learning
performance of the population by instructing more effective
and efficient social imitations.
B. UT2004 and POGAMUT
Platform. Over the past few decades, modern computer
games have attracted dramatic attention and garnered a large
part of the multi-billion dollar computer game industry. In
particular, the modeling of non-player characters (NPCs) is of
great significance for the success of commercial video games,
as it can improve the playability of games and the satisfaction
of human players [37], [38]. Taking this cue, Unreal Tournament
2004 (UT2004), a well-known FPS shooter game developed
by Epic Games, was used to test the performance of the
proposed method in creating autonomous AIs in a complex
game environment.
Figure 14 shows a screen snapshot of UT2004. The player
uses a keyboard and mouse to control the actions of a virtual
character (or agent) in a 3D limited virtual world. When the
game server receives a specific command from the player, the
agent's operations are executed. These actions are primitive:
they mainly include running, jumping, shooting, and changing
weapons. UT2004 has many built-in game scenes. The
most common are Deathmatch, Domination, and Capture
The Flag (CTF). Since UT2004 provides an environment for
virtual agents, it is widely used to test the performance of
intelligent agents.
In UT2004, players have a number of high-level tasks, such
as killing opponents and collecting items. In Deathmatch, the
goal of each player is to kill their opponents, and the first player
or team who obtains a predefined game score wins the game.
When a player dies, the player reenters the game with
100 health points and two basic weapons. Therefore, players
70
50
30
Forward Imitation
Backward Imitation
Bi-MeMAS
10
500
1,000
Learning Trails
FIGURE 11 Success rate of agents under Forward Imitation, Backward
Imitation, and Bi-MeMAS in completing MNT missions.
1,500
2,000
130
170
10
50
90
500
Forward Imitation
Backward Imitation
Bi-MeMAS
1,000
Learning Trails
FIGURE 12 Memotype number of agents under Forward Imitation,
Backward Imitation, and Bi-MeMAS in completing MNT missions.
12
3
6
9
Forward
Imitation
Backward
Imitation
Bi-MeMAS
FIGURE 13 Imitation number of agents under Forward Imitation,
Backward Imitation, and Bi-MeMAS in completing MNT missions.
×104
1,500
2,000
NOVEMBER 2021 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 65
Memotype Number
Success Rate
Imitation Number
IEEE Computational Intelligence Magazine - November 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - November 2021
