IEEE Robotics & Automation Magazine - December 2021 - 31

Acknowledgment
This work was supported by the National Natural Science
Foundation of China (grants 62173228, U1764264, and
61873165). Ming Yang is the corresponding author.
References
[1] Y. L. Murphey, R. Milton, and L. Kiliaris, " Driver's style classification
using jerk analysis, " in Proc. 2009 IEEE Workshop Comput. Intell.
Veh. Veh. Syst., pp. 23-28. doi: 10.1109/CIVVS.2009.4938719.
[2] D. Dörr, D. Grabengiesser, and F. Gauterin, " Online driving style recognition
using fuzzy logic, " in Proc. 17th Int. IEEE Conf. Intell. Transport. Syst.
(ITSC), 2014, pp. 1021-1026. doi: 10.1109/ITSC.2014.6957822.
[3] L. Xu, J. Hu, H. Jiang, and W. Meng, " Establishing style-oriented driver
models by imitating human driving behaviors, " IEEE Trans. Intell. Transp.
Syst., vol. 16, no. 5, pp. 2522-2530, 2015. doi: 10.1109/TITS.2015.2409870.
[4] M. Bojarski et al., " End to end learning for self-driving cars, " 2016,
arXiv:1604.07316.
[5] H. Xu, Y. Gao, F. Yu, and T. Darrell, " End-to-end learning of driving models
from large-scale video datasets, " in Proc. IEEE Conf. Comput. Vision Pattern
Recognit. (CVPR), July 2017, pp. 3530-3538. doi: 10.1109/CVPR.2017.376.
[6] K. Min, H. Kim, and K. Hub, " Deep Q learning based high level
driving policy determination, " in Proc. 2018 IEEE Intell. Veh. Symp.
(IV), pp. 226-231. doi: 10.1109/IVS.2018.8500645.
[7] V. Mnih et al., " Human-level control through deep reinforcement
learning, " Nature, vol. 518, no. 7540, p. 529, 2015. doi: 10.1038/nature14236.
[8] Z. Wang, T. Schaul, M. Hessel, H. Van Hasselt, M. Lanctot, and N. De
Freitas, " Dueling network architectures for deep reinforcement learning, "
2015, arXiv:1511.06581.
[9] H. Van Hasselt, A. Guez, and D. Silver, " Deep reinforcement learning
with double Q-learning, " in Proc. AAAI Conf. Artif. Intell., Feb.
2016, vol. 2, pp. 2094-2100.
[10] H. Van Seijen, M. Fatemi, J. Romoff, R. Laroche, T. Barnes, and J.
Tsang, " Hybrid reward architecture for reinforcement learning, " in
Proc. Adv. Neural Inf. Process. Syst., 2017, pp. 5392-5402.
[11] A. E. Sallab, M. Abdou, E. Perot, and S. Yogamani, " End-to-end deep
reinforcement learning for lane keeping assist, " 2016, arXiv:1612.04340.
[12] T. P. Lillicrap et al., " Continuous control with deep reinforcement
learning, " 2015, arXiv:1509.02971.
[13] W. Yuan, M. Yang, Y. He, C. Wang, and B. Wang, " Multi-reward
architecture based reinforcement learning for highway driving policies, "
in Proc. 2019 IEEE Intell. Transport. Syst. Conf. (ITSC), pp. 3810-
3815. doi: 10.1109/ITSC.2019.8917304.
[14] R. Zheng, C. Liu, and Q. Guo, " A decision-making method for
autonomous vehicles based on simulation and reinforcement learning, "
in Proc. 2013 IEEE Int. Conf. Mach. Learn. Cybern., vol. 1, pp. 362-
369. doi: 10.1109/ICMLC.2013.6890495.
[15] A. E. Sallab, M. Abdou, E. Perot, and S. Yogamani, " Deep reinforcement
learning framework for autonomous driving, " Electron. Imag.,
vol. 2017, no. 19, pp. 70-76, 2017. doi: 10.2352/ISSN.2470-1173.2017.19.AVM-023.
[16] C. J. Watkins and P. Dayan, " Q-learning, " Mach. Learn., vol. 8, nos.
3-4, pp. 279-292, 1992. doi: 10.1023/A:1022676722315.
[17] P. Wang, C.-Y. Chan, and A. de La Fortelle, " A reinforcement learning
based approach for automated lane change maneuvers, " in Proc. 2018
IEEE Intell. Veh. Symp. (IV), pp. 1379-1384. doi: 10.1109/IVS.2018.8500556.
[18] A. Yu, R. Palefsky-Smith, and R. Bedi, " Deep reinforcement learning
for simulated autonomous vehicle control, " Stanford University, Stanford,
CA, Winter 2016. [Online]. Available: http://cs231n.stanford.edu/
reports/2016/pdfs/112_Report.pdf
[19] C.-J. Hoel, K. Wolff, and L. Laine, " Tactical decision-making in
autonomous driving by reinforcement learning with uncertainty estimation, "
in Proc. 2020 IEEE Intell. Veh. Symp. (IV), pp. 1563-1569. doi:
10.1109/IV47402.2020.9304614.
[20] P. Wolf, K. Kurzer, T. Wingert, F. Kuhnt, and J. M. Zollner, " Adaptive
behavior generation for autonomous driving using deep reinforcement
learning with compact semantic states, " in Proc. 2018 IEEE Intell.
Veh. Symp. (IV), pp. 993-1000. doi: 10.1109/IVS.2018.8500427.
[21] R. S. Sutton et al., " Horde: A scalable real-time architecture for learning
knowledge from unsupervised sensorimotor interaction, " in Proc.
10th Int. Conf. Auton. Agents Multiagent Syst., 2011, vol. 2, pp. 761-768.
[22] T. Schaul, D. Horgan, K. Gregor, and D. Silver, " Universal
value function approximators, " in Proc. Int. Conf. Mach. Learn.,
2015, pp. 1312-1320.
[23] L.-J. Lin, " Self-improving reactive agents based on reinforcement
learning, planning and teaching, " Mach. Learn., vol. 8, nos. 3-4,
pp. 293-321, 1992. doi: 10.1007/BF00992699.
[24] V. Mnih et al., " Playing Atari with deep reinforcement learning, "
2013, arXiv:1312.5602.
[25] J. Schmidhuber, " Curious model-building control systems, " in Proc.
1991 IEEE Int. Joint Conf. Neural Netw., Singapore, pp. 1458-1463. doi:
10.1109/IJCNN.1991.170605.
[26] L. A. Atherton, D. Dupret, and J. R. Mellor, " Memory trace replay:
The shaping of memory consolidation by neuromodulation, " Trends
Neurosci., vol. 38, no. 9, pp. 560-570, 2015. doi: 10.1016/j.tins.2015.07.004.
[27] H. F. Ólafsdóttir, C. Barry, A. B. Saleem, D. Hassabis, and H. J. Spiers,
" Hippocampal place cells construct reward related sequences through
unexplored space, " Elife, vol. 4, p. e06063, 2015. doi: 10.7554/eLife.06063.
[28] T. Schaul, J. Quan, I. Antonoglou, and D. Silver, " Prioritized experience
replay, " 2015, arXiv:1511.05952.
Wei Yuan, University of Michigan-Shanghai Jiao Tong University
Joint Institute, Shanghai Jiao Tong University, Shanghai,
200240, China. Email: weiy1991@sjtu.edu.cn.
Yueyuan Li, Department of Automation, Shanghai Jiao Tong
University, Key Laboratory of System Control and Information
Processing, Ministry of Education of China, Shanghai, 200240,
China. Email: rowena_lee@sjtu.edu.cn.
Hanyang Zhuang, University of Michigan-Shanghai Jiao Tong
University Joint Institute, Shanghai Jiao Tong University,
Shanghai, 200240, China. Email: zhuanghany11@sjtu.edu.cn.
Chunxiang Wang, Department of Automation, Shanghai Jiao
Tong University, Key Laboratory of System Control and
Information Processing, Ministry of Education of China,
Shanghai, 200240, China. Email: wangcx@sjtu.edu.cn.
Ming Yang, Department of Automation, Shanghai Jiao Tong
University, Key Laboratory of System Control and Information
Processing, Ministry of Education of China, Shanghai,
200240, China. Email: mingyang@sjtu.edu.cn.
DECEMBER 2021 * IEEE ROBOTICS & AUTOMATION MAGAZINE *
31
http://cs231n.stanford.edu/reports/2016/pdfs/112_Report.pdf http://cs231n.stanford.edu/reports/2016/pdfs/112_Report.pdf
IEEE Robotics & Automation Magazine - December 2021

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - December 2021
