IEEE Robotics & Automation Magazine - December 2022 - 103

Acknowledgment
This project/research has received funding from the European
Union's Horizon 2020 Framework Program for Research
and Innovation under Specific Grant Agreement (SGA)
Number 945539 (Human Brain Project SGA3). This work
was also supported by the National Natural Science Foundation
of China (grant 61902442), Pazhou Lab PZL2021KF0020 and
Shenzhen Basic Research Grant JCYJ20180508152434975.
The corresponding author is Long Cheng. This article has
supplementary downloadable material available at https://doi.
org/10.1109/MRA.2022.3204237, provided by the authors.
References
[1] M. Tesch et al., " Parameterized and scripted gaits for modular snake
robots, " Adv. Robot., vol. 23, no. 9, pp. 1131-1158, 2009, doi:
10.1163/156855309X452566.
[2] S. Hirose, Biologically Inspired Robots: Snake-Like Locomotors and
Manipulators, vol. 1093. London, U.K.: Oxford Univ. Press, 1993.
[3] Z. Bing, L. Cheng, G. Chen, F. Röhrbein, K. Huang, and A. Knoll,
" Towards autonomous locomotion: CPG-based control of smooth 3d
slithering gait transition of a snake-like robot, " Bioinspiration Biomimetics,
vol. 12, no. 3, p. 35,001, Apr. 2017, doi: 10.1088/1748-3190/aa644c.
[4] Z. Bing, L. Cheng, K. Huang, M. Zhou, and A. Knoll, " CPG-based
control of smooth transition for body shape and locomotion speed of a
snake-like robot, " in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), pp.
4146-4153, doi: 10.1109/ICRA.2017.7989476.
[5] P. Liljebäck, K. Y. Pettersen, Ø. Stavdahl, and J. T. Gravdahl, Snake
Robots: Modelling, Mechatronics, and Control. London: Springer Science
& Business Media, 2012.
[6] Z. Bing, C. Lemke, Z. Jiang, K. Huang, and A. Knoll, " Energy-efficient
slithering gait exploration for a snake-like robot based on
reinforcement learning, " in Proc. 28th Int. Joint Conf. Artif. Intell.,
Macao, China: International Joint Conferences on Artificial Intelligence
Organization, Aug. 2019, pp. 1-7.
[7] X. B. Peng, M. Andrychowicz, W. Zaremba, and P. Abbeel, " Sim-toreal
transfer of robotic control with dynamics randomization, " in Proc.
IEEE Int. Conf. Robot. Autom. (ICRA), Brisbane, QLD, Australia, May
2018, pp. 3803-3810, doi: 10.1109/ICRA.2018.8460528.
[8] A. Crespi and A. J. Ijspeert, " Online optimization of swimming and
crawling in an amphibious snake robot, " IEEE Trans. Robot., vol. 24, no.
1, pp. 75-87, Feb. 2008, doi: 10.1109/TRO.2008.915426.
[9] M. Tesch, J. Schneider, and H. Choset, " Using response surfaces and
expected improvement to optimize snake robot gait parameters, " in
Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., Sep. 2011, pp. 1069-1074,
doi: 10.1109/IROS.2011.6095076.
[10] S. Ouyang and W. Wei, " Flexible adaptive control of snake-like
robot based on LSTM and gait, " J. Phys., Conf. Ser., vol. 1487, no. 1, p.
012049, 2020, doi: 10.1088/1742-6596/1487/1/012049.
[11] A. Nagabandi, G. Kahn, R. S. Fearing, and S. Levine, " Neural network
dynamics for model-based deep reinforcement learning with
model-free fine-tuning, " in Proc. IEEE Int. Conf. Robot. Autom. (ICRA),
May 2018, pp. 7559-7566, doi: 10.1109/ICRA.2018.8463189.
[12] Y. Nakamura, T. Mori, M-a Sato, and S. Ishii, " Reinforcement
learning for a biped robot based on a CPG-actor-critic method, " Neural
Netw., vol. 20, no. 6, pp. 723-735, 2007, doi: 10.1016/j.neunet.
2007.01.002.
[13] J. Shi, T. Dear, and S. D. Kelly, " Deep reinforcement learning for
snake robot locomotion, " IFAC-PapersOnLine, vol. 53, no. 2, pp. 9688-
9695, 2020, doi: 10.1016/j.ifacol.2020.12.2619.
[14] S. Fukunaga, Y. Nakamura, K. Aso, and S. Ishii, " Reinforcement
learning for a snake-like robot controlled by a central pattern generator, "
in Proc. IEEE Conf. Robot., Autom. Mechatronics, 2004, vol. 2, pp.
909-914, doi: 10.1109/RAMECH.2004.1438039.
[15] X. Liu, R. Gasoto, Z. Jiang, C. Onal, and J. Fu, " Learning to locomote
with artificial neural-network and CPG-based control in a soft
snake robot, " in Proc. 2020 IEEE/RSJ Int. Conf. Intell. Robots Syst.
(IROS), pp. 7758-7765, doi: 10.1109/IROS45743.2020.9340763.
[16] S. Chatterjee et al., " Reinforcement learning approach to generate
goal-directed locomotion of a snake-like robot with screw-drive units, "
in Proc. 2014 23rd Int. Conf. Robot. Alpe-Adria-Danube Region
(RAAD), pp. 1-7, doi: 10.1109/RAAD.2014.7002234.
[17] G. Sartoretti, Y. Shi, W. Paivine, M. Travers, and H. Choset, " Distributed
learning for the decentralized control of articulated mobile
robots, " in Proc. 2018 IEEE Int. Conf. Robot. Autom. (ICRA), pp. 3789-
3794, doi: 10.1109/ICRA.2018.8460802.
[18] T. Haarnoja, S. Ha, A. Zhou, J. Tan, G. Tucker, and S. Levine,
" Learning to walk via deep reinforcement learning, " in Proc. Robot.,
Sci. Syst., Freiburg, Jun. 2019, doi: 10.15607/RSS.2019.XV.011.
[19] T. Miki, J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter,
" Learning robust perceptive locomotion for quadrupedal robots in the wild, "
Sci. Robot., vol. 7, no. 62, p. eabk2822, 2022, doi: 10.1126/scirobotics.abk2822.
[20] P. Christiano et al., " Transfer from Simulation to real world
through learning deep inverse dynamics model, " Oct. 2016, arXiv:1610.
03518 [cs].
[21] X. B. Peng, M. Andrychowicz, W. Zaremba, and P. Abbeel, " Sim-toreal
transfer of robotic control with dynamics randomization, " in Proc.
2018 IEEE Int. Conf. Robot. Autom. (ICRA), pp. 3803-3810, doi: 10.1109/
ICRA.2018.8460528.
[22] M. Wulfmeier, I. Posner, and P. Abbeel, " Mutual alignment transfer
learning, " in Proc. Conf. Robot Learn., 2017, pp. 281-290.
[23] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov,
" Proximal policy optimization algorithms, " 2017, arXiv:1707.06347.
[24] J. Hwangbo et al., " Learning agile and dynamic motor skills for
legged robots, " Sci. Robot., vol. 4, no. 26, p. eaau5872, Jan. 2019, doi:
10.1126/scirobotics.aau5872.
Zhenshan Bing, Department of Informatics, Technical University
of Munich, Munich, 85748, Germany. Email: bing@
in.tum.de.
Long Cheng, College of Computer Science and Artificial Intelligence,
Wenzhou University, Wenzhou, 325035, China, and
Shenzhen Institute, Sun Yat-sen University, Guangzhou
543001, China. Email: chenglong@wzu.edu.cn.
Kai Huang, School of Data and Computer Science, Sun Yat-sen
University, Guangzhou 543001, China, and Pazhou Lab, Guangzhou,
543001, China. Email: huangk36@mail.sysu.edu.cn.
Alois Knoll, Department of Informatics, Technical University
of Munich, Munich 85748, Germany. Email knoll@in.tum.de.
DECEMBER 2022 * IEEE ROBOTICS & AUTOMATION MAGAZINE *
103
https://doi.org/10.1109/mra.2022.3204327 https://doi.org/10.1109/mra.2022.3204327
IEEE Robotics & Automation Magazine - December 2022

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2022
