IEEE Robotics & Automation Magazine - September 2017 - 108
By Mustafa Ersen, Erhan Oztop, and Sanem Sariel
Cognition-Enabled
Robot Manipulation
in Human Environments
S
ervice robots are expected to
play an important role in our
daily lives as our companions in
home and work environments in
the near future. An important requirement for fulfilling this expectation is to
equip robots with skills to perform everyday
manipulation tasks, the success of which is
crucial for most home chores, such as cooking,
cleaning, and shopping. Robots have been
used successfully for manipulation tasks in wellstructured and controlled factory environments for
decades. Designing skills for robots working in uncontrolled
human environments raises many potential challenges in
various subdisciplines, such as computer vision, automated
planning, and human-robot interaction. In spite of the recent
progress in these fields, there are still challenges to tackle.
This article outlines problems in different research areas
related to mobile manipulation from the cognitive perspective, reviews recently published works and the stateof-the-art approaches to address these problems, and
discusses open problems to be solved to realize robot assistants that can be used in manipulation tasks in unstructured
human environments.
IMAGE LICENSED BY INGRAM PUBLISHING
Requirements, Recent Work, and Open Problems
Digital Object Identifier 10.1109/MRA.2016.2616538
Date of publication: 7 April 2017
108
*
IEEE ROBOTICS & AUTOMATION MAGAZINE
*
SEPTEMBER 2017
1070-9932/17©2017IEEE
Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2017
IEEE Robotics & Automation Magazine - September 2017 - Cover1
IEEE Robotics & Automation Magazine - September 2017 - Cover2
IEEE Robotics & Automation Magazine - September 2017 - 1
IEEE Robotics & Automation Magazine - September 2017 - 2
IEEE Robotics & Automation Magazine - September 2017 - 3
IEEE Robotics & Automation Magazine - September 2017 - 4
IEEE Robotics & Automation Magazine - September 2017 - 5
IEEE Robotics & Automation Magazine - September 2017 - 6
IEEE Robotics & Automation Magazine - September 2017 - 7
IEEE Robotics & Automation Magazine - September 2017 - 8
IEEE Robotics & Automation Magazine - September 2017 - 9
IEEE Robotics & Automation Magazine - September 2017 - 10
IEEE Robotics & Automation Magazine - September 2017 - 11
IEEE Robotics & Automation Magazine - September 2017 - 12
IEEE Robotics & Automation Magazine - September 2017 - 13
IEEE Robotics & Automation Magazine - September 2017 - 14
IEEE Robotics & Automation Magazine - September 2017 - 15
IEEE Robotics & Automation Magazine - September 2017 - 16
IEEE Robotics & Automation Magazine - September 2017 - 17
IEEE Robotics & Automation Magazine - September 2017 - 18
IEEE Robotics & Automation Magazine - September 2017 - 19
IEEE Robotics & Automation Magazine - September 2017 - 20
IEEE Robotics & Automation Magazine - September 2017 - 21
IEEE Robotics & Automation Magazine - September 2017 - 22
IEEE Robotics & Automation Magazine - September 2017 - 23
IEEE Robotics & Automation Magazine - September 2017 - 24
IEEE Robotics & Automation Magazine - September 2017 - 25
IEEE Robotics & Automation Magazine - September 2017 - 26
IEEE Robotics & Automation Magazine - September 2017 - 27
IEEE Robotics & Automation Magazine - September 2017 - 28
IEEE Robotics & Automation Magazine - September 2017 - 29
IEEE Robotics & Automation Magazine - September 2017 - 30
IEEE Robotics & Automation Magazine - September 2017 - 31
IEEE Robotics & Automation Magazine - September 2017 - 32
IEEE Robotics & Automation Magazine - September 2017 - 33
IEEE Robotics & Automation Magazine - September 2017 - 34
IEEE Robotics & Automation Magazine - September 2017 - 35
IEEE Robotics & Automation Magazine - September 2017 - 36
IEEE Robotics & Automation Magazine - September 2017 - 37
IEEE Robotics & Automation Magazine - September 2017 - 38
IEEE Robotics & Automation Magazine - September 2017 - 39
IEEE Robotics & Automation Magazine - September 2017 - 40
IEEE Robotics & Automation Magazine - September 2017 - 41
IEEE Robotics & Automation Magazine - September 2017 - 42
IEEE Robotics & Automation Magazine - September 2017 - 43
IEEE Robotics & Automation Magazine - September 2017 - 44
IEEE Robotics & Automation Magazine - September 2017 - 45
IEEE Robotics & Automation Magazine - September 2017 - 46
IEEE Robotics & Automation Magazine - September 2017 - 47
IEEE Robotics & Automation Magazine - September 2017 - 48
IEEE Robotics & Automation Magazine - September 2017 - 49
IEEE Robotics & Automation Magazine - September 2017 - 50
IEEE Robotics & Automation Magazine - September 2017 - 51
IEEE Robotics & Automation Magazine - September 2017 - 52
IEEE Robotics & Automation Magazine - September 2017 - 53
IEEE Robotics & Automation Magazine - September 2017 - 54
IEEE Robotics & Automation Magazine - September 2017 - 55
IEEE Robotics & Automation Magazine - September 2017 - 56
IEEE Robotics & Automation Magazine - September 2017 - 57
IEEE Robotics & Automation Magazine - September 2017 - 58
IEEE Robotics & Automation Magazine - September 2017 - 59
IEEE Robotics & Automation Magazine - September 2017 - 60
IEEE Robotics & Automation Magazine - September 2017 - 61
IEEE Robotics & Automation Magazine - September 2017 - 62
IEEE Robotics & Automation Magazine - September 2017 - 63
IEEE Robotics & Automation Magazine - September 2017 - 64
IEEE Robotics & Automation Magazine - September 2017 - 65
IEEE Robotics & Automation Magazine - September 2017 - 66
IEEE Robotics & Automation Magazine - September 2017 - 67
IEEE Robotics & Automation Magazine - September 2017 - 68
IEEE Robotics & Automation Magazine - September 2017 - 69
IEEE Robotics & Automation Magazine - September 2017 - 70
IEEE Robotics & Automation Magazine - September 2017 - 71
IEEE Robotics & Automation Magazine - September 2017 - 72
IEEE Robotics & Automation Magazine - September 2017 - 73
IEEE Robotics & Automation Magazine - September 2017 - 74
IEEE Robotics & Automation Magazine - September 2017 - 75
IEEE Robotics & Automation Magazine - September 2017 - 76
IEEE Robotics & Automation Magazine - September 2017 - 77
IEEE Robotics & Automation Magazine - September 2017 - 78
IEEE Robotics & Automation Magazine - September 2017 - 79
IEEE Robotics & Automation Magazine - September 2017 - 80
IEEE Robotics & Automation Magazine - September 2017 - 81
IEEE Robotics & Automation Magazine - September 2017 - 82
IEEE Robotics & Automation Magazine - September 2017 - 83
IEEE Robotics & Automation Magazine - September 2017 - 84
IEEE Robotics & Automation Magazine - September 2017 - 85
IEEE Robotics & Automation Magazine - September 2017 - 86
IEEE Robotics & Automation Magazine - September 2017 - 87
IEEE Robotics & Automation Magazine - September 2017 - 88
IEEE Robotics & Automation Magazine - September 2017 - 89
IEEE Robotics & Automation Magazine - September 2017 - 90
IEEE Robotics & Automation Magazine - September 2017 - 91
IEEE Robotics & Automation Magazine - September 2017 - 92
IEEE Robotics & Automation Magazine - September 2017 - 93
IEEE Robotics & Automation Magazine - September 2017 - 94
IEEE Robotics & Automation Magazine - September 2017 - 95
IEEE Robotics & Automation Magazine - September 2017 - 96
IEEE Robotics & Automation Magazine - September 2017 - 97
IEEE Robotics & Automation Magazine - September 2017 - 98
IEEE Robotics & Automation Magazine - September 2017 - 99
IEEE Robotics & Automation Magazine - September 2017 - 100
IEEE Robotics & Automation Magazine - September 2017 - 101
IEEE Robotics & Automation Magazine - September 2017 - 102
IEEE Robotics & Automation Magazine - September 2017 - 103
IEEE Robotics & Automation Magazine - September 2017 - 104
IEEE Robotics & Automation Magazine - September 2017 - 105
IEEE Robotics & Automation Magazine - September 2017 - 106
IEEE Robotics & Automation Magazine - September 2017 - 107
IEEE Robotics & Automation Magazine - September 2017 - 108
IEEE Robotics & Automation Magazine - September 2017 - 109
IEEE Robotics & Automation Magazine - September 2017 - 110
IEEE Robotics & Automation Magazine - September 2017 - 111
IEEE Robotics & Automation Magazine - September 2017 - 112
IEEE Robotics & Automation Magazine - September 2017 - 113
IEEE Robotics & Automation Magazine - September 2017 - 114
IEEE Robotics & Automation Magazine - September 2017 - 115
IEEE Robotics & Automation Magazine - September 2017 - 116
IEEE Robotics & Automation Magazine - September 2017 - 117
IEEE Robotics & Automation Magazine - September 2017 - 118
IEEE Robotics & Automation Magazine - September 2017 - 119
IEEE Robotics & Automation Magazine - September 2017 - 120
IEEE Robotics & Automation Magazine - September 2017 - 121
IEEE Robotics & Automation Magazine - September 2017 - 122
IEEE Robotics & Automation Magazine - September 2017 - 123
IEEE Robotics & Automation Magazine - September 2017 - 124
IEEE Robotics & Automation Magazine - September 2017 - 125
IEEE Robotics & Automation Magazine - September 2017 - 126
IEEE Robotics & Automation Magazine - September 2017 - 127
IEEE Robotics & Automation Magazine - September 2017 - 128
IEEE Robotics & Automation Magazine - September 2017 - 129
IEEE Robotics & Automation Magazine - September 2017 - 130
IEEE Robotics & Automation Magazine - September 2017 - 131
IEEE Robotics & Automation Magazine - September 2017 - 132
IEEE Robotics & Automation Magazine - September 2017 - 133
IEEE Robotics & Automation Magazine - September 2017 - 134
IEEE Robotics & Automation Magazine - September 2017 - 135
IEEE Robotics & Automation Magazine - September 2017 - 136
IEEE Robotics & Automation Magazine - September 2017 - 137
IEEE Robotics & Automation Magazine - September 2017 - 138
IEEE Robotics & Automation Magazine - September 2017 - 139
IEEE Robotics & Automation Magazine - September 2017 - 140
IEEE Robotics & Automation Magazine - September 2017 - 141
IEEE Robotics & Automation Magazine - September 2017 - 142
IEEE Robotics & Automation Magazine - September 2017 - 143
IEEE Robotics & Automation Magazine - September 2017 - 144
IEEE Robotics & Automation Magazine - September 2017 - 145
IEEE Robotics & Automation Magazine - September 2017 - 146
IEEE Robotics & Automation Magazine - September 2017 - 147
IEEE Robotics & Automation Magazine - September 2017 - 148
IEEE Robotics & Automation Magazine - September 2017 - 149
IEEE Robotics & Automation Magazine - September 2017 - 150
IEEE Robotics & Automation Magazine - September 2017 - 151
IEEE Robotics & Automation Magazine - September 2017 - 152
IEEE Robotics & Automation Magazine - September 2017 - 153
IEEE Robotics & Automation Magazine - September 2017 - 154
IEEE Robotics & Automation Magazine - September 2017 - 155
IEEE Robotics & Automation Magazine - September 2017 - 156
IEEE Robotics & Automation Magazine - September 2017 - 157
IEEE Robotics & Automation Magazine - September 2017 - 158
IEEE Robotics & Automation Magazine - September 2017 - 159
IEEE Robotics & Automation Magazine - September 2017 - 160
IEEE Robotics & Automation Magazine - September 2017 - 161
IEEE Robotics & Automation Magazine - September 2017 - 162
IEEE Robotics & Automation Magazine - September 2017 - 163
IEEE Robotics & Automation Magazine - September 2017 - 164
IEEE Robotics & Automation Magazine - September 2017 - 165
IEEE Robotics & Automation Magazine - September 2017 - 166
IEEE Robotics & Automation Magazine - September 2017 - 167
IEEE Robotics & Automation Magazine - September 2017 - 168
IEEE Robotics & Automation Magazine - September 2017 - 169
IEEE Robotics & Automation Magazine - September 2017 - 170
IEEE Robotics & Automation Magazine - September 2017 - 171
IEEE Robotics & Automation Magazine - September 2017 - 172
IEEE Robotics & Automation Magazine - September 2017 - 173
IEEE Robotics & Automation Magazine - September 2017 - 174
IEEE Robotics & Automation Magazine - September 2017 - 175
IEEE Robotics & Automation Magazine - September 2017 - 176
IEEE Robotics & Automation Magazine - September 2017 - 177
IEEE Robotics & Automation Magazine - September 2017 - 178
IEEE Robotics & Automation Magazine - September 2017 - 179
IEEE Robotics & Automation Magazine - September 2017 - 180
IEEE Robotics & Automation Magazine - September 2017 - 181
IEEE Robotics & Automation Magazine - September 2017 - 182
IEEE Robotics & Automation Magazine - September 2017 - 183
IEEE Robotics & Automation Magazine - September 2017 - 184
IEEE Robotics & Automation Magazine - September 2017 - Cover3
IEEE Robotics & Automation Magazine - September 2017 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2010
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2010
https://www.nxtbookmedia.com