IEEE Robotics & Automation Magazine - June 2023 - 86
©SHUTTERSTOCK.COM/PHONLAMAI PHOTO
Tumbling Robot
Control Using
Reinforcement
Learning
An Adaptive Control Policy
That Transfers Well to the Real World
By Andrew Schwartzwald, Matthew Tlachac, Luis Guzman,
Athanasios Bacharis, and Nikolaos Papanikolopoulos
Tumbling robots are simple platforms that are able to traverse
large obstacles relative to their size, at the cost of being difficult
to control. Existing control methods apply only a subset of
possible robot motions and make the assumption of flat terrain.
Reinforcement learning (RL) allows for the development
of sophisticated control schemes that can adapt to diverse
environments. By utilizing domain randomization while training
in simulation, a robust control policy can be learned that
transfers well to the real world. In this article, we implement
Digital Object Identifier 10.1109/MRA.2022.3188215
Date of current version: 29 July 2022
86
IEEE ROBOTICS & AUTOMATION MAGAZINE JUNE 2023
autonomous set point navigation on a tumbling robot prototype
and evaluate it on flat, uneven, and valley-hill terrain.
Our results demonstrate that RL-based control policies can
generalize well to challenging environments that were not
encountered during training. The flexibility of our system
demonstrates the viability of nontraditional robots for navigational
tasks.
TUMBLING LOCOMOTION
In recent years, many distinct tumbling robot platforms have
been designed. This class of robots uses gravity to assist their
movement by rolling their body through a given terrain. These
1070-9932/22©2022IEEE
http://www.SHUTTERSTOCK.COM/PHONLAMAI
https://orcid.org/0000-0002-2177-1870
IEEE Robotics & Automation Magazine - June 2023
Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - June 2023
Contents
IEEE Robotics & Automation Magazine - June 2023 - Cover1
IEEE Robotics & Automation Magazine - June 2023 - Cover2
IEEE Robotics & Automation Magazine - June 2023 - Contents
IEEE Robotics & Automation Magazine - June 2023 - 2
IEEE Robotics & Automation Magazine - June 2023 - 3
IEEE Robotics & Automation Magazine - June 2023 - 4
IEEE Robotics & Automation Magazine - June 2023 - 5
IEEE Robotics & Automation Magazine - June 2023 - 6
IEEE Robotics & Automation Magazine - June 2023 - 7
IEEE Robotics & Automation Magazine - June 2023 - 8
IEEE Robotics & Automation Magazine - June 2023 - 9
IEEE Robotics & Automation Magazine - June 2023 - 10
IEEE Robotics & Automation Magazine - June 2023 - 11
IEEE Robotics & Automation Magazine - June 2023 - 12
IEEE Robotics & Automation Magazine - June 2023 - 13
IEEE Robotics & Automation Magazine - June 2023 - 14
IEEE Robotics & Automation Magazine - June 2023 - 15
IEEE Robotics & Automation Magazine - June 2023 - 16
IEEE Robotics & Automation Magazine - June 2023 - 17
IEEE Robotics & Automation Magazine - June 2023 - 18
IEEE Robotics & Automation Magazine - June 2023 - 19
IEEE Robotics & Automation Magazine - June 2023 - 20
IEEE Robotics & Automation Magazine - June 2023 - 21
IEEE Robotics & Automation Magazine - June 2023 - 22
IEEE Robotics & Automation Magazine - June 2023 - 23
IEEE Robotics & Automation Magazine - June 2023 - 24
IEEE Robotics & Automation Magazine - June 2023 - 25
IEEE Robotics & Automation Magazine - June 2023 - 26
IEEE Robotics & Automation Magazine - June 2023 - 27
IEEE Robotics & Automation Magazine - June 2023 - 28
IEEE Robotics & Automation Magazine - June 2023 - 29
IEEE Robotics & Automation Magazine - June 2023 - 30
IEEE Robotics & Automation Magazine - June 2023 - 31
IEEE Robotics & Automation Magazine - June 2023 - 32
IEEE Robotics & Automation Magazine - June 2023 - 33
IEEE Robotics & Automation Magazine - June 2023 - 34
IEEE Robotics & Automation Magazine - June 2023 - 35
IEEE Robotics & Automation Magazine - June 2023 - 36
IEEE Robotics & Automation Magazine - June 2023 - 37
IEEE Robotics & Automation Magazine - June 2023 - 38
IEEE Robotics & Automation Magazine - June 2023 - 39
IEEE Robotics & Automation Magazine - June 2023 - 40
IEEE Robotics & Automation Magazine - June 2023 - 41
IEEE Robotics & Automation Magazine - June 2023 - 42
IEEE Robotics & Automation Magazine - June 2023 - 43
IEEE Robotics & Automation Magazine - June 2023 - 44
IEEE Robotics & Automation Magazine - June 2023 - 45
IEEE Robotics & Automation Magazine - June 2023 - 46
IEEE Robotics & Automation Magazine - June 2023 - 47
IEEE Robotics & Automation Magazine - June 2023 - 48
IEEE Robotics & Automation Magazine - June 2023 - 49
IEEE Robotics & Automation Magazine - June 2023 - 50
IEEE Robotics & Automation Magazine - June 2023 - 51
IEEE Robotics & Automation Magazine - June 2023 - 52
IEEE Robotics & Automation Magazine - June 2023 - 53
IEEE Robotics & Automation Magazine - June 2023 - 54
IEEE Robotics & Automation Magazine - June 2023 - 55
IEEE Robotics & Automation Magazine - June 2023 - 56
IEEE Robotics & Automation Magazine - June 2023 - 57
IEEE Robotics & Automation Magazine - June 2023 - 58
IEEE Robotics & Automation Magazine - June 2023 - 59
IEEE Robotics & Automation Magazine - June 2023 - 60
IEEE Robotics & Automation Magazine - June 2023 - 61
IEEE Robotics & Automation Magazine - June 2023 - 62
IEEE Robotics & Automation Magazine - June 2023 - 63
IEEE Robotics & Automation Magazine - June 2023 - 64
IEEE Robotics & Automation Magazine - June 2023 - 65
IEEE Robotics & Automation Magazine - June 2023 - 66
IEEE Robotics & Automation Magazine - June 2023 - 67
IEEE Robotics & Automation Magazine - June 2023 - 68
IEEE Robotics & Automation Magazine - June 2023 - 69
IEEE Robotics & Automation Magazine - June 2023 - 70
IEEE Robotics & Automation Magazine - June 2023 - 71
IEEE Robotics & Automation Magazine - June 2023 - 72
IEEE Robotics & Automation Magazine - June 2023 - 73
IEEE Robotics & Automation Magazine - June 2023 - 74
IEEE Robotics & Automation Magazine - June 2023 - 75
IEEE Robotics & Automation Magazine - June 2023 - 76
IEEE Robotics & Automation Magazine - June 2023 - 77
IEEE Robotics & Automation Magazine - June 2023 - 78
IEEE Robotics & Automation Magazine - June 2023 - 79
IEEE Robotics & Automation Magazine - June 2023 - 80
IEEE Robotics & Automation Magazine - June 2023 - 81
IEEE Robotics & Automation Magazine - June 2023 - 82
IEEE Robotics & Automation Magazine - June 2023 - 83
IEEE Robotics & Automation Magazine - June 2023 - 84
IEEE Robotics & Automation Magazine - June 2023 - 85
IEEE Robotics & Automation Magazine - June 2023 - 86
IEEE Robotics & Automation Magazine - June 2023 - 87
IEEE Robotics & Automation Magazine - June 2023 - 88
IEEE Robotics & Automation Magazine - June 2023 - 89
IEEE Robotics & Automation Magazine - June 2023 - 90
IEEE Robotics & Automation Magazine - June 2023 - 91
IEEE Robotics & Automation Magazine - June 2023 - 92
IEEE Robotics & Automation Magazine - June 2023 - 93
IEEE Robotics & Automation Magazine - June 2023 - 94
IEEE Robotics & Automation Magazine - June 2023 - 95
IEEE Robotics & Automation Magazine - June 2023 - 96
IEEE Robotics & Automation Magazine - June 2023 - 97
IEEE Robotics & Automation Magazine - June 2023 - 98
IEEE Robotics & Automation Magazine - June 2023 - 99
IEEE Robotics & Automation Magazine - June 2023 - 100
IEEE Robotics & Automation Magazine - June 2023 - 101
IEEE Robotics & Automation Magazine - June 2023 - 102
IEEE Robotics & Automation Magazine - June 2023 - 103
IEEE Robotics & Automation Magazine - June 2023 - 104
IEEE Robotics & Automation Magazine - June 2023 - 105
IEEE Robotics & Automation Magazine - June 2023 - 106
IEEE Robotics & Automation Magazine - June 2023 - 107
IEEE Robotics & Automation Magazine - June 2023 - 108
IEEE Robotics & Automation Magazine - June 2023 - 109
IEEE Robotics & Automation Magazine - June 2023 - 110
IEEE Robotics & Automation Magazine - June 2023 - 111
IEEE Robotics & Automation Magazine - June 2023 - 112
IEEE Robotics & Automation Magazine - June 2023 - 113
IEEE Robotics & Automation Magazine - June 2023 - 114
IEEE Robotics & Automation Magazine - June 2023 - 115
IEEE Robotics & Automation Magazine - June 2023 - 116
IEEE Robotics & Automation Magazine - June 2023 - 117
IEEE Robotics & Automation Magazine - June 2023 - 118
IEEE Robotics & Automation Magazine - June 2023 - 119
IEEE Robotics & Automation Magazine - June 2023 - 120
IEEE Robotics & Automation Magazine - June 2023 - 121
IEEE Robotics & Automation Magazine - June 2023 - 122
IEEE Robotics & Automation Magazine - June 2023 - 123
IEEE Robotics & Automation Magazine - June 2023 - 124
IEEE Robotics & Automation Magazine - June 2023 - 125
IEEE Robotics & Automation Magazine - June 2023 - 126
IEEE Robotics & Automation Magazine - June 2023 - 127
IEEE Robotics & Automation Magazine - June 2023 - 128
IEEE Robotics & Automation Magazine - June 2023 - 129
IEEE Robotics & Automation Magazine - June 2023 - 130
IEEE Robotics & Automation Magazine - June 2023 - 131
IEEE Robotics & Automation Magazine - June 2023 - 132
IEEE Robotics & Automation Magazine - June 2023 - 133
IEEE Robotics & Automation Magazine - June 2023 - 134
IEEE Robotics & Automation Magazine - June 2023 - 135
IEEE Robotics & Automation Magazine - June 2023 - 136
IEEE Robotics & Automation Magazine - June 2023 - Cover3
IEEE Robotics & Automation Magazine - June 2023 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2010
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2010
https://www.nxtbookmedia.com