IEEE Computational Intelligence Magazine - November 2021 - 69

intensity of the controller, a method was proposed to analyze
the clarity of useful memory (known as memotypes in agents'
minds) for remembering the information captured from different
sensors. Furthermore, inspired by the human-like social
learning in MeMAS, a meme selection process driven by a bidirectional
imitation strategy in meme external evolution was proposed.
In this method, an agent decides whether to request imitation
from others based on its estimation of the state uncertainty.
When receiving a request, the selected teacher agent makes a
response based on its confidence on the state importance. Lastly,
the performance efficacy of the proposed method with intensity
attenuation control and bidirectional imitation was verified carefully
via comprehensive experimental studies on a widely used
MNT platform and a well-known first-person shooter game,
namely UT2004.
The current study utilized the memetic automaton as the
backbone of the multi-agent
learning framework, where
memes were conceptualized as cognitive neurons stored in
the agent's mind universe. However, falling back on the original
definition of a meme as a basic unit of cultural information,
there remains much potential for exploring the
generality and applicability of meme computation. For
instance, alternative computational forms of memetic representations,
i.e, Bayesian models, can be explored based on different
assumptions and considerations about the agent
environment. More efficient ways to estimate the effectiveness
or importance of meme knowledge could be derived from
specific domain instructions.
Acknowledgments
This work was supported in part by the National Natural Science
Foundation of China under Grant 61906032, the NSFCLiaoning
Province United Foundation under Grant U1908214,
the Fundamental Research Funds for the Central Universities
under grant DUT21TD107, the LiaoNing Revitalization
Talents Program, No. XLYC2008017, the National Key
Research and Development Program of China under Grant
2018YFC0910500, the National Natural Science Foundation of
China under Grant 61976034, and the Liaoning Key Research
and Development Program under Grant 2019JH2/10100030.
References
[1] A. Gupta and Y.-S. Ong, Memetic Computation: The Mainspring of Knowledge Transfer
in a Data-Driven Optimization Era. New York, NY, USA: Springer-Verlag, 2018, vol. 21.
[2] F. Neri and C. Cotta, " Memetic algorithms and memetic computing optimization:
A literature review, " Swarm and Evol. Comput., vol. 2, pp. 1-14, 2012. doi: 10.1016/j.
swevo.2011.11.003.
[3] A. Gupta and Y.-S. Ong, " The memetic automaton, " in Memetic Computation. New
York, NY, USA: Springer-Verlag, 2019, pp. 47-61.
[4] X. Chen, Y.-S. Ong, M.-H. Lim, and K. C. Tan, " A multi-facet survey on memetic
computation, " IEEE Trans. Evol. Comput., vol. 15, no. 5, pp. 591-607, 2011. doi: 10.1109/
TEVC.2011.2132725.
[5] A. O. Ibrahim, S. M. Shamsuddin, A. Abraham, and S. N. Qasem, " Adaptive memetic
method of multi-objective genetic evolutionary algorithm for backpropagation neural network, "
Neural Comput. Appl., vol. 31, no. 9, pp. 4945-4962, 2019. doi: 10.1007/s00521018-03990-0.
[6]
P. R. Lorenzo and J. Nalepa, " Memetic evolution of deep neural networks, " in Proc.
Genetic Evol. Comput. Conf., 2018, pp. 505-512.
[7] L. Feng, Y.-S. Ong, A.-H. Tan, and I. W. Tsang, " Memes as building blocks: a case
study on evolutionary optimization+ transfer learning for routing problems, " Memetic
Comput., vol. 7, no. 3, pp. 159-180, 2015. doi: 10.1007/s12293-015-0166-x.
[8] Y. Hou, Y. Zeng, and Y. S. Ong, " A memetic multi-agent demonstration learning approach
with behavior prediction, " in Proc. 15th Int. Conf. Autonom. Agents and Multiagent
Syst., 2016, pp. 539-547.
[9] Y. Hou, Y.-S. Ong, J. Tang, and Y. Zeng, " Evolutionary multiagent transfer learning
with model-based opponent behavior prediction, " IEEE Trans. Syst., Man, Cybernetics:
Syst., early access, Dec. 27, 2019. doi: 10.1109/TSMC.2019.2958846.
[10] L. Feng, Y.-S. Ong, A.-H. Tan, and X.-S. Chen, " Towards human-like social multiagents
with memetic automaton, " in Proc. IEEE Congr. Evol. Comput. (CEC), 2011, pp.
1092-1099.
[11] Y. Hou, Y.-S. Ong, L. Feng, and J. M. Zurada, " An evolutionary transfer reinforcement
learning framework for multiagent systems, " IEEE Trans. Evol. Comput., vol. 21, no.
4, pp. 601-615, 2017. doi: 10.1109/TEVC.2017.2664665.
[12] D. Wang, A.-H. Tan, C. Miao, and A. A. Moustafa, " Modelling autobiographical
memory loss across life span, " in Proc. AAAI Conf. Artif. Intell., vol. 33, no. 01, 2019, pp.
1368-1375. doi: 10.1609/aaai.v33i01.33011368.s.
[13] C. Brown, Cognitive Psychology. Newbury Park, CA, USA: Sage, 2006.
[14] M. Fischer, M. Moscovitch, and C. Alain, " Incidental auditory learning and memory-guided
attention: Examining the role of attention at the behavioural and neural level
using EEG, " Neuropsychologia, vol. 147, p. 107586, 2020.
[15] J. Driver, " A selective review of selective attention research from the past century, "
Brit. J. Psychol., vol. 92, no. 1, pp. 53-78, 2001. doi: 10.1348/000712601162103.
[16] B. Shinn-Cunningham, V. Best, A. Kingstone, J. Fawcett, and E. Risko, " Auditory
selective attention, " Handbook of Attention, vol. 99, 2015.
[17] A. Gupta, Y.-S. Ong, and L. Feng, " Insights on transfer optimization: Because experience
is the best teacher, " IEEE Trans. Emerg. Topics Comput. Intell., vol. 2, no. 1, pp.
51-64, 2017. doi: 10.1109/TETCI.2017.2769104.
[18] L. Feng, Y.-S. Ong, M.-H. Lim, and I. W. Tsang, " Memetic search with interdomain
learning: A realization between CVRP and CARP, " IEEE Trans. Evol. Comput., vol. 19,
no. 5, pp. 644-658, 2014. doi: 10.1109/TEVC.2014.2362558.
[19] L. Feng, Y.-S. Ong, S. Jiang, and A. Gupta, " Autoencoding evolutionary search
with learning across heterogeneous problems, " IEEE Trans. Evol. Comput., vol. 21, no. 5,
pp. 760-772, 2017. doi: 10.1109/TEVC.2017.2682274.
[20] K. C. Tan, L. Feng, and M. Jiang, " Evolutionary transfer optimization-a new frontier
in evolutionary computation research, " IEEE Comput. Intell. Mag., vol. 16, no. 1,
pp. 22-33, 2021. doi: 10.1109/MCI.2020.3039066.
[21] L. Nunes and E. Oliveira, " On learning by exchanging advice, " 2002, cs/0203010.
[22] R. Dawkins, The Selfish Gene. London, U.K.: Oxford Univ. Press, 2016.
[23] Y. Zeng, X. Chen, Y.-S. Ong, J. Tang, and Y. Xiang, " Structured memetic automation
for online human-like social behavior learning, " IEEE Trans. Evol. Comput., vol. 21,
no. 1, pp. 102-115, 2016. doi: 10.1109/TEVC.2016.2577593.
[24] A.-H. Tan, " Falcon: A fusion architecture for learning, cognition, and navigation, "
in Proc. IEEE Int. Joint Conf. Neural Netw. (IEEE Cat. No. 04CH37541), 2004, vol. 4,
pp. 3297-3302.
[25] A.-H. Tan, N. Lu, and D. Xiao, " Integrating temporal difference methods and selforganizing
neural networks for reinforcement learning with delayed evaluative feedback, "
IEEE Trans. Neural Netw., vol. 19, no. 2, pp. 230-244, 2008.
[26] A.-H. Tan, B. Subagdja, D. Wang, and L. Meng, " Self-organizing neural networks
for universal learning and multimodal memory encoding, " Neural Netw., vol. 120, pp. 58-
73, 2019. doi: 10.1016/j.neunet.2019.08.020.
[27] D. E. Broadbent, Perception and Communication. Amsterdam, The Netherlands: Elsevier,
2013.
[28] A. Lazaric, " Transfer in reinforcement learning: a framework and a survey, " in Reinforcement
Learning. New York, NY, USA: Springer-Verlag, 2012, pp. 143-173.
[29] K. Mo, Y. Zhang, S. Li, J. Li, and Q. Yang, " Personalizing a dialogue system with
transfer reinforcement learning, " in Proc. AAAI Conf. Artif. Intell., 2018, vol. 32, no. 1.
[30] E. Ilhan, J. Gow, and D. Perez-Liebana, " Teaching on a budget in multi-agent deep
reinforcement learning, " in Proc. IEEE Conf. Games (CoG), 2019, pp. 1-8. doi: 10.1109/
CIG.2019.8847988.
[31] F. L. Da Silva and A. H. R. Costa, " A survey on transfer learning for multiagent
reinforcement learning systems, " J. Artif. Intell. Res., vol. 64, pp. 645-703, 2019. doi:
10.1613/jair.1.11396.
[32] B. D. Argall, S. Chernova, M. Veloso, and B. Browning, " A survey of robot learning
from demonstration, " Robot. Autonom. Syst., vol. 57, no. 5, pp. 469-483, 2009. doi:
10.1016/j.robot.2008.10.024.
[33] F. L. Da Silva, R. Glatt, and A. H. R. Costa, " Simultaneously learning and advising
in multiagent reinforcement learning, " in Proc. 16th Conf. Autonom. Agents Multiagent Syst.,
2017, pp. 1100-1108.
[34] F. L. Da Silva, P. Hernandez-Leal, B. Kartal, and M. E. Taylor, " Uncertainty-aware
action advising for deep reinforcement learning agents, " in Proc. AAAI Conf. Artif. Intell.,
vol. 34, no. 04, 2020, pp. 5792-5799. doi: 10.1609/aaai.v34i04.6036.
[35] T. Liu, B. Tian, Y. Ai, L. Li, D. Cao, and F.-Y. Wang, " Parallel reinforcement learning:
A framework and case study, " IEEE/CAA J. Automat. Sinica, vol. 5, no. 4, pp. 827-
835, 2018. doi: 10.1109/JAS.2018.7511144.
[36] J. Kober and J. Peters, " Imitation and reinforcement learning, " IEEE Robot. Automat.
Mag., vol. 17, no. 2, pp. 55-62, 2010. doi: 10.1109/MRA.2010.936952.
[37] E. Novak and J. Tassell, " Video games that improve " learning to learn " : Focus on
action video game play elements, " in Proc. IEEE 17th Int. Conf. Adv. Learning Technol.
(ICALT), 2017, pp. 142-144. doi: 10.1109/ICALT.2017.152.
[38] C. Pelling and H. Gardner, " Two human-like imitation-learning bots with probabilistic
behaviors, " in Proc. IEEE Conf. Games (CoG), 2019, pp. 1-7. doi: 10.1109/
CIG.2019.8847995.
NOVEMBER 2021 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 69
IEEE Computational Intelligence Magazine - November 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - November 2021
