IEEE Robotics & Automation Magazine - June 2023 - 85

[91] F. Muratore, T. Gruner, F. Wiese, B. Belousov, M. Gienger, and J. Peters, " Neural
posterior domain randomization, " in Proc. 5th Conf. Robot Learn., 2022, pp. 1532-1542.
[92] F. Muratore, F. Ramos, G. Turk, W. Yu, M. Gienger, and J. Peters, " Robot
learning from randomized simulations: A review, " Frontiers Robot. AI, vol.
9, p. 799,893, Apr. 2022, doi: 10.3389/frobt.2022.799893.
[93] V. Myers, E. Biyik, N. Anari, and D. Sadigh, " Learning multimodal
rewards from rankings, " in Proc. 5th Conf. Robot Learn., 2022, pp. 342-352.
[94] O. Nachum, M. Ahn, H. Ponte, S. Gu, and V. Kumar, " Multi-agent manipulation via
locomotion using hierarchical sim2real, " in Proc. Conf. Robot Learn., 2020, pp. 110-121.
[95] O. Nachum, S. Gu, H. Lee, and S. Levine, " Data-efficient hierarchical reinforcement
learning, " arXiv:1805.08296.
[96] S. Narvekar, B. Peng, M. Leonetti, J. Sinapov, M. E. Taylor, and P. Stone,
" Curriculum learning for reinforcement learning domains: A framework and survey, "
J. Mach. Learn. Res., vol. 21, no. 181, pp. 1-50, Jan. 2020.
[97] G. Ning, X. Zhang, and H. Liao, " Autonomic robotic ultrasound imaging system
based on reinforcement learning, " IEEE Trans. Biomed. Eng., vol. 68, no. 9,
pp. 2787-2797, Sep. 2021, doi: 10.1109/TBME.2021.3054413.
[98] OpenAI et al., " Solving Rubik's cube with a robot hand, " arXiv:1910.07113.
[99] T. Osa, J. Pajarinen, G. Neumann, J. A. Bagnell, P. Abbeel, and J. Peters, " An
algorithmic perspective on imitation learning, " Found. Trends® Robot., vol. 7, nos.
1-2, pp. 1-179, Mar. 2018, doi: 10.1561/2300000053.
[100] J. Parker-Holder et al., " Automated reinforcement learning (AutoRL): A survey
and open problems, " J. Artif. Int. Res., vol. 74, Aug. 2022, pp. 517-568.
[101] S. Pateria, B. Subagdja, A.-h. Tan, and C. Quek, " Hierarchical reinforcement
learning: A comprehensive survey, " ACM Comput. Surv., vol. 54, no. 5, pp. 1-35,
Jun. 2021, doi: 10.1145/3453160.
[102] X. B. Peng, M. Andrychowicz, W. Zaremba, and P. Abbeel, " Sim-to-real
transfer of robotic control with dynamics randomization, " in Proc. 2018 IEEE Int.
Conf. Robot. Automat. (ICRA), pp. 3803-3810, doi: 10.1109/ICRA.2018.8460528.
[103] X. B. Peng, G. Berseth, K. Yin, and M. Van De Panne, " DeepLoco:
Dynamic locomotion skills using hierarchical deep reinforcement learning, " vol.
36, no. 4, pp. 1-13, Aug. 2017, doi: 10.1145/3072959.3073602.
[104] X. B. Peng, E. Coumans, T. Zhang, T.-W. Edward Lee, J. Tan, and S. Levine,
" Learning agile robotic locomotion skills by imitating animals, " in Proc. Robot.,
Sci. Syst., vol. 7, 2020. [Online]. Available: https://openreview.net /
forum?id=IajZhOef9Qw
[105] X. B. Peng, Z. Ma, P. Abbeel, S. Levine, and A. Kanazawa, " AMP: Adversarial
motion priors for stylized physics-based character control, " ACM Trans. Graph., vol.
40, no. 4, pp. 1-20, Aug. 2021, doi: 10.1145/3476576.3476723.
[106] K. Pertsch, Y. Lee, and J. Lim, " Accelerating reinforcement learning with
learned skill priors, " in Proc. Conf. Robot Learn., 2021, pp. 188-204.
[107] K. Pertsch, Y. Lee, Y. Wu, and J. J. Lim, " Demonstration-guided reinforcement
learning with learned skills, " in Proc. 5th Conf. Robot Learn., 2021, pp. 729-739.
[108] A. Prakash et al., " Structured domain randomization: Bridging the reality
gap by context-aware synthetic data, " in Proc. 2019 Int. Conf. Robot. Automat.
(ICRA), pp. 7249-7255, doi: 10.1109/ICRA.2019.8794443.
[109] F. Ramos, R. Possas, and D. Fox, " BayesSim: Adaptive domain randomization
via probabilistic inference for robotics simulators, " in Proc. Robot., Sci. Syst.,
A. Bicchi, H. Kress-Gazit, and S. Hutchinson, Eds. 2019. [Online]. Available:
http://www.roboticsproceedings.org/rss15/p29.pd
[110] K. Rao, C. Harris, A. Irpan, S. Levine, J. Ibarz, and M. Khansari,
" RL-CycleGan: Reinforcement learning aware simulation-to-real, " in Proc. IEEE/
CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2020, pp. 11,157-11,166.
[111] D. Rodriguez and S. Behnke, " DeepWalk: Omnidirectional bipedal gait by
deep reinforcement learning, " in Proc. 2021 IEEE Int. Conf. Robot. Automat.
(ICRA), pp. 3033-3039, doi: 10.1109/ICRA48506.2021.9561717.
[112] S. Ross, G. Gordon, and D. Bagnell, " A reduction of imitation learning and
structured prediction to no-regret online learning, " in Proc. 14th Int. Conf. Artif.
Intell. Statist., 2011, pp. 627-635.
[113] N. Rudin, D. Hoeller, P. Reist, and M. Hutter, " Learning to walk in minutes
using massively parallel deep reinforcement learning, " in Proc. 5th Conf. Robot
Learn., 2022, pp. 91-100.
[114] T. Salimans, J. Ho, X. Chen, S. Sidor, and I. Sutskever, " Evolution strategies
as a scalable alternative to reinforcement learning, " 2017, arXiv:1703.03864.
[115] S. Stefan, " Learning from demonstration, " in Proc. Adv. Neural Inf.
Process. Syst., 1996, pp. 1040-1046.
[116] J. Schrittwieser, T. Hubert, A. Mandhane, M. Barekatain, I. Antonoglou, and
D. Silver, " Online and offline reinforcement learning by planning with a learned
model, " in Proc. Adv. Neural Inf. Process. Syst., Apr. 2021, pp. 27580-27591.
[117] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, " Proximal
policy optimization algorithms, " 2017, arXiv:1707.06347.
[118] A. Sharma, A. Gupta, S. Levine, K. Hausman, and C. Finn, " Autonomous
reinforcement learning via subgoal curricula, " in Proc. Adv. Neural Inf. Process.
Syst., 2021, pp. 1-13.
[119] J. Siekmann, Y. Godse, A. Fern, and J. Hurst, " Sim-to-real learning of all common
bipedal gaits via periodic reward composition, " in Proc. 2021 IEEE Int. Conf.
Robot. Automat. (ICRA), pp. 7309-7315, doi: 10.1109/ICRA48506.2021.9561814.
[120] P. Soviany, R. T. Ionescu, P. Rota, and N. Sebe, " Curriculum learning: A survey, "
Int. J. Comput. Vis., vol. 130, no. 6, pp. 1-40, Jun. 2022, doi: 10.1007/s11263-022-01611-x.
[121] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction.
Cambridge, MA, USA: MIT Press, 2018.
[122] R. S. Sutton, D. Precup, and S. Singh, " Between MDPS and semi-MDPS: A
framework for temporal abstraction in reinforcement learning, " Artif. Intell., vol.
112, nos. 1-2, pp. 181-211, Aug. 1999, doi: 10.1016/S0004-3702(99)00052-1.
[123] J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel,
" Domain randomization for transferring deep neural networks from simulation to
the real world, " in Proc. 2017 IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), pp.
23-30, doi: 10.1109/IROS.2017.8202133.
[124] Y.-Y. Tsai, H. Xu, Z. Ding, C. Zhang, E. Johns, and B. Huang, " DROID:
Minimizing the reality gap using single-shot human demonstration, " IEEE Robot.
Automat. Lett., vol. 6, no. 2, pp. 3168-3175, Apr. 2021, doi: 10.1109/LRA.2021.3062311.
[125] P. Varin, L. Grossman, and S. Kuindersma, " A comparison of action spaces
for learning manipulation tasks, " in Proc. 2019 IEEE/RSJ Int. Conf. Intell. Robots
Syst. (IROS), pp. 6015-6021, doi: 10.1109/IROS40897.2019.8967946.
[126] L. von Rueden et al., " Informed machine learning - A taxonomy and survey
of integrating prior knowledge into learning systems, " IEEE Trans. Knowl. Data
Eng., early access, 2021, doi: 10.1109/TKDE.2021.3079836.
[127] L. von Rueden, S. Mayer, R. Sifa, C. Bauckhage, and J. Garcke, " Combining
machine learning and simulation to a hybrid modelling approach: Current and
future directions, " in Proc. Int. Symp. Intell. Data Anal., Springer-Verlag, 2020, pp.
548-560, doi: 10.1007/978-3-030-44584-3_43.
[128] J. Wang, Y. Liu, and B. Li, " Reinforcement learning with perturbed
rewards, " in Proc. AAAI Conf. Artif. Intell., Apr. 2020, vol. 34, no. 4, pp. 6202-
6209, doi: 10.1609/aaai.v34i04.6086.
[129] L. Wang, X. Meng, Y. Xiang, and D. Fox, " Hierarchical policies for cluttered-scene
grasping with latent plans, " IEEE Robot. Automat. Lett., vol. 7, no. 2,
pp. 2883-2890, Apr. 2022, doi: 10.1109/LRA.2022.3143198.
[130] L. Wang, Y. Xiang, and D. Fox, " Manipulation trajectory optimization with
online grasp synthesis and selection, " in Proc. Robot., Sci. Syst. (RSS), 2020, pp. 1-10.
[131] L. Wang, Y. Xiang, W. Yang, A. Mousavian, and D. Fox, " Goal-auxiliary
actor-critic for 6d robotic grasping with point clouds, " in Proc. 5th Conf. Robot
Learn., 2022, pp. 70-80.
[132] M. Wang and W. Deng, " Deep visual domain adaptation: A survey, "
Neurocomputing, vol. 312, pp. 135-153, Oct. 2018, doi: 10.1016/j.neucom.2018.05.083.
[133] Z. Wang et al., " Critic regularized regression, " in Proc. Adv. Neural Inf.
Process. Syst., 2020, pp. 7768-7778.
[134] W. Whitney, R. Agarwal, K. Cho, and A. Gupta, " Dynamics-aware embeddings, "
in Proc. Int. Conf. Learn. Representations, 2020. [Online]. Available:
https://openreview.net/forum?id=BJgZGeHFPH
[135] N. Wilde, E. Biyik, D. Sadigh, and S. L. Smith, " Learning reward functions from
scale feedback, " in Proc. 5th Conf. Robot Learn., 2022, pp. 353-362.
[136] J. Wong, V. Makoviychuk, A. Anandkumar, and Y. Zhu, " OSCAR: Datadriven
operational space control for adaptive and robust robot manipulation, " in
Proc. 2022 Int. Conf. Robot. Automat. (ICRA), pp. 10,519-10,526.
[137] F. Xia, A. R. Zamir, Z. He, A. Sax, J. Malik, and S. Savarese, " Gibson env:
Real-world perception for embodied agents, " in Proc. IEEE Conf. Comput. Vis.
Pattern Recognit. (CVPR), Jun. 2018, pp. 9068-9079, doi: 10.1109/CVPR.2018.00945.
[138] H. Xu, Y. R. Wang, S. Eppel, A. Aspuru-Guzik, F. Shkurti, and A. Garg,
" Seeing glass: Joint point-cloud and depth completion for transparent objects, " in
Proc. 5th Conf. Robot Learn., 2022, pp. 827-838.
[139] Z. Yang, K. Merrick, L. Jin, and H. A. Abbass, " Hierarchical deep reinforcement
learning for continuous action control, " IEEE Trans. Neural Netw. Learn. Syst.,
vol. 29, no. 11, pp. 5174-5184, Nov. 2018, doi: 10.1109/TNNLS.2018.2805379.
[140] D. Yarats et al., " Don't change the algorithm, change the data: Exploratory
data for offline reinforcement learning, " in Proc. ICLR Workshop Generalizable
Policy Learn. Phys. World, Jan. 2022. [Online]. Available: https://openreview.net/
forum?id=Su-zh4a41Z5
[141] W. Yu, J. Tan, C. K. Liu, and G. Turk, " Preparing for the unknown:
Learning a universal policy with online system identification, " in Proc. Robot.,
Sci. Syst. XIII, Cambridge, MA, USA: MIT Press, Jul. 12-16, 2017, pp. 1-10.
[142] Y. Yu, " Towards sample efficient reinforcement learning, " in Proc. 27th Int.
Joint Conf. Artif. Intell. (IJCAI'18), AAAI Press, 2018, pp. 5739-5743.
[143] J. Zhang et al., " VR-goggles for robots: Real-to-sim domain adaptation for
visual control, " IEEE Robot. Automat. Lett., vol. 4, no. 2, pp. 1148-1155, Apr.
2019, doi: 10.1109/LRA.2019.2894216.
[144] W. Zhao, J. P. Queralta, and T. Westerlund, " Sim-to-real transfer in deep reinforcement
learning for robotics: A survey, " in Proc. 2020 IEEE Symp. Series
Comput. Intell. (SSCI), pp. 737-744, doi: 10.1109/SSCI47803.2020.9308468.
JUNE 2023 IEEE ROBOTICS & AUTOMATION MAGAZINE
85
IEEE Robotics & Automation Magazine - June 2023

Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2023
