IEEE Robotics & Automation Magazine - September 2017 - 53
imperfect sensing and localization as well as in environmental
dynamics because of imprecise control and unpredictability
about when security events may occur. This challenge leads
us to examine state-of-the-art formalisms for modeling
robots' task dynamics and for planning under uncertainty-
formalisms that push the boundaries of robot autonomy.
The primary contribution of this work, however, is the integration of complementary research technologies from video
surveillance, mobile robotics, and AI. We demonstrate the
efficacy of our integration through a prototype system that
includes a small number of robots and cameras deployed in
realistic environments. A modular architecture and general
framework for representing and communicating surveillance
events makes our system a useful test bed for experimenting with various research technologies. In contrast to past
results in multirobot patrolling that employed human operators
to orchestrate the robots' behavior [7], we are able to achieve
fully autonomous security robots capable of making decisions
on their own, with the potential to help human operators.
Overview of the Surveillance Framework
We begin with a brief overview of our framework, which is
motivated by a concrete example of a decision faced by a
patrolling robot. This leads us to characterize the decisionmaking problem as well as to structure our system in support
of the implementation and testing of decision-theoretic planning for mobile surveillance robots.
Motivating Example
Imagine adding a robot to the observation environment
shown in Figure 1. In contrast to the static cameras placed at
fixed positions, the robot is capable of dynamically patrolling the building. It can move from room to room, using its
39
36
33
Soccer Field
30
27
24
Meters
observing our parking lots, university campuses, and shopping
malls. Imagine the monotonous patrols of armies of security
guards through countless corridors. Deliberate over the difficult strategic decisions of where and how to allocate precious
human resources, both in response to immediate security concerns and in anticipation of future conditions. To maintain
safety and security, the conventional surveillance system relies
critically on human attention, action, and intelligence. However, such reliance is untenable in a society where the trend is
toward more cameras embedded in larger and more complex
environments to defend against a growing array of potential
threats (from burglary to natural disasters to terrorist attacks).
Here, we advocate a shift toward reliance on autonomous system components so that society may scale up to meet presentday surveillance needs.
One aspect of this topic that has received considerable
attention from researchers is real-time scene analysis. Systems
have already been developed to autonomously analyze video
streams in environments such as transportation networks [6],
[27] and public spaces [5] to identify actors and characterize
their behavior. Recent examples include IBM's Smart Surveillance System project [22] and Yao et al.'s system for cooperative object tracking [30]. There are also approaches for
activity interpretation [8], [12], [13], [20], [25], while other
works are more focused on meeting low-bandwidth requirements by locally processing video-feed images [4]. Although
these systems can autonomously extract relevant information
for surveillance purposes, they are still heavily dependent on a
team of human security personnel to perform such actions as
covering areas that may be outside the range of the stationary
sensor network and resolving situations that may require
physical intervention. Our work aims to increase autonomy
and to reduce the human burden by introducing autonomous
mobile robots into the system.
Research in robot mobility has advanced to the point that
robots now have the ability to navigate complex environments,
patrolling as human guards would. Equipped with cameras
and other sensors, they can also serve as mobile surveillance
nodes to augment a network of statically situated cameras. For
instance, a robot can provide temporary coverage of areas that
may become critical because of camera failures or other anomalies. Moreover, robots have the mobility, sensors, and actuators to respond directly to events detected over fixed camera
streams, thereby leveraging real-time scene analysis.
To integrate these complementary research technologies
effectively and to render robots truly autonomous requires a
third key technology: intelligent decision making. Robots
should choose their actions so as to fulfill a combination of
objectives, given limited resources. This is often framed as a
robot task selection and allocation problem [10] and has been
approached through a variety of AI techniques, from logicbased (classical) planning methods [9], to market-based
(auction) solutions [15] and those relying on constraint optimization [16]. An obstacle to applying such techniques
here is that surveillance decisions are riddled with uncertainty. Uncertainty is present in robots' awareness because of
Mobile
Robotics
Lab
Elevator
Hallway
21
18
15
East
Corridor
12
9
North Corridor
6
Coffee
Room
3
3
6
9 12 15 18 21 24 27 30 33 36 39
Meters
Figure 1. A staged indoor surveillance environment with the
positions of the static cameras (red circles) and the common
coordinate system for event location.
SEPTEMBER 2017
*
IEEE ROBOTICS & AUTOMATION MAGAZINE
*
53
Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2017
IEEE Robotics & Automation Magazine - September 2017 - Cover1
IEEE Robotics & Automation Magazine - September 2017 - Cover2
IEEE Robotics & Automation Magazine - September 2017 - 1
IEEE Robotics & Automation Magazine - September 2017 - 2
IEEE Robotics & Automation Magazine - September 2017 - 3
IEEE Robotics & Automation Magazine - September 2017 - 4
IEEE Robotics & Automation Magazine - September 2017 - 5
IEEE Robotics & Automation Magazine - September 2017 - 6
IEEE Robotics & Automation Magazine - September 2017 - 7
IEEE Robotics & Automation Magazine - September 2017 - 8
IEEE Robotics & Automation Magazine - September 2017 - 9
IEEE Robotics & Automation Magazine - September 2017 - 10
IEEE Robotics & Automation Magazine - September 2017 - 11
IEEE Robotics & Automation Magazine - September 2017 - 12
IEEE Robotics & Automation Magazine - September 2017 - 13
IEEE Robotics & Automation Magazine - September 2017 - 14
IEEE Robotics & Automation Magazine - September 2017 - 15
IEEE Robotics & Automation Magazine - September 2017 - 16
IEEE Robotics & Automation Magazine - September 2017 - 17
IEEE Robotics & Automation Magazine - September 2017 - 18
IEEE Robotics & Automation Magazine - September 2017 - 19
IEEE Robotics & Automation Magazine - September 2017 - 20
IEEE Robotics & Automation Magazine - September 2017 - 21
IEEE Robotics & Automation Magazine - September 2017 - 22
IEEE Robotics & Automation Magazine - September 2017 - 23
IEEE Robotics & Automation Magazine - September 2017 - 24
IEEE Robotics & Automation Magazine - September 2017 - 25
IEEE Robotics & Automation Magazine - September 2017 - 26
IEEE Robotics & Automation Magazine - September 2017 - 27
IEEE Robotics & Automation Magazine - September 2017 - 28
IEEE Robotics & Automation Magazine - September 2017 - 29
IEEE Robotics & Automation Magazine - September 2017 - 30
IEEE Robotics & Automation Magazine - September 2017 - 31
IEEE Robotics & Automation Magazine - September 2017 - 32
IEEE Robotics & Automation Magazine - September 2017 - 33
IEEE Robotics & Automation Magazine - September 2017 - 34
IEEE Robotics & Automation Magazine - September 2017 - 35
IEEE Robotics & Automation Magazine - September 2017 - 36
IEEE Robotics & Automation Magazine - September 2017 - 37
IEEE Robotics & Automation Magazine - September 2017 - 38
IEEE Robotics & Automation Magazine - September 2017 - 39
IEEE Robotics & Automation Magazine - September 2017 - 40
IEEE Robotics & Automation Magazine - September 2017 - 41
IEEE Robotics & Automation Magazine - September 2017 - 42
IEEE Robotics & Automation Magazine - September 2017 - 43
IEEE Robotics & Automation Magazine - September 2017 - 44
IEEE Robotics & Automation Magazine - September 2017 - 45
IEEE Robotics & Automation Magazine - September 2017 - 46
IEEE Robotics & Automation Magazine - September 2017 - 47
IEEE Robotics & Automation Magazine - September 2017 - 48
IEEE Robotics & Automation Magazine - September 2017 - 49
IEEE Robotics & Automation Magazine - September 2017 - 50
IEEE Robotics & Automation Magazine - September 2017 - 51
IEEE Robotics & Automation Magazine - September 2017 - 52
IEEE Robotics & Automation Magazine - September 2017 - 53
IEEE Robotics & Automation Magazine - September 2017 - 54
IEEE Robotics & Automation Magazine - September 2017 - 55
IEEE Robotics & Automation Magazine - September 2017 - 56
IEEE Robotics & Automation Magazine - September 2017 - 57
IEEE Robotics & Automation Magazine - September 2017 - 58
IEEE Robotics & Automation Magazine - September 2017 - 59
IEEE Robotics & Automation Magazine - September 2017 - 60
IEEE Robotics & Automation Magazine - September 2017 - 61
IEEE Robotics & Automation Magazine - September 2017 - 62
IEEE Robotics & Automation Magazine - September 2017 - 63
IEEE Robotics & Automation Magazine - September 2017 - 64
IEEE Robotics & Automation Magazine - September 2017 - 65
IEEE Robotics & Automation Magazine - September 2017 - 66
IEEE Robotics & Automation Magazine - September 2017 - 67
IEEE Robotics & Automation Magazine - September 2017 - 68
IEEE Robotics & Automation Magazine - September 2017 - 69
IEEE Robotics & Automation Magazine - September 2017 - 70
IEEE Robotics & Automation Magazine - September 2017 - 71
IEEE Robotics & Automation Magazine - September 2017 - 72
IEEE Robotics & Automation Magazine - September 2017 - 73
IEEE Robotics & Automation Magazine - September 2017 - 74
IEEE Robotics & Automation Magazine - September 2017 - 75
IEEE Robotics & Automation Magazine - September 2017 - 76
IEEE Robotics & Automation Magazine - September 2017 - 77
IEEE Robotics & Automation Magazine - September 2017 - 78
IEEE Robotics & Automation Magazine - September 2017 - 79
IEEE Robotics & Automation Magazine - September 2017 - 80
IEEE Robotics & Automation Magazine - September 2017 - 81
IEEE Robotics & Automation Magazine - September 2017 - 82
IEEE Robotics & Automation Magazine - September 2017 - 83
IEEE Robotics & Automation Magazine - September 2017 - 84
IEEE Robotics & Automation Magazine - September 2017 - 85
IEEE Robotics & Automation Magazine - September 2017 - 86
IEEE Robotics & Automation Magazine - September 2017 - 87
IEEE Robotics & Automation Magazine - September 2017 - 88
IEEE Robotics & Automation Magazine - September 2017 - 89
IEEE Robotics & Automation Magazine - September 2017 - 90
IEEE Robotics & Automation Magazine - September 2017 - 91
IEEE Robotics & Automation Magazine - September 2017 - 92
IEEE Robotics & Automation Magazine - September 2017 - 93
IEEE Robotics & Automation Magazine - September 2017 - 94
IEEE Robotics & Automation Magazine - September 2017 - 95
IEEE Robotics & Automation Magazine - September 2017 - 96
IEEE Robotics & Automation Magazine - September 2017 - 97
IEEE Robotics & Automation Magazine - September 2017 - 98
IEEE Robotics & Automation Magazine - September 2017 - 99
IEEE Robotics & Automation Magazine - September 2017 - 100
IEEE Robotics & Automation Magazine - September 2017 - 101
IEEE Robotics & Automation Magazine - September 2017 - 102
IEEE Robotics & Automation Magazine - September 2017 - 103
IEEE Robotics & Automation Magazine - September 2017 - 104
IEEE Robotics & Automation Magazine - September 2017 - 105
IEEE Robotics & Automation Magazine - September 2017 - 106
IEEE Robotics & Automation Magazine - September 2017 - 107
IEEE Robotics & Automation Magazine - September 2017 - 108
IEEE Robotics & Automation Magazine - September 2017 - 109
IEEE Robotics & Automation Magazine - September 2017 - 110
IEEE Robotics & Automation Magazine - September 2017 - 111
IEEE Robotics & Automation Magazine - September 2017 - 112
IEEE Robotics & Automation Magazine - September 2017 - 113
IEEE Robotics & Automation Magazine - September 2017 - 114
IEEE Robotics & Automation Magazine - September 2017 - 115
IEEE Robotics & Automation Magazine - September 2017 - 116
IEEE Robotics & Automation Magazine - September 2017 - 117
IEEE Robotics & Automation Magazine - September 2017 - 118
IEEE Robotics & Automation Magazine - September 2017 - 119
IEEE Robotics & Automation Magazine - September 2017 - 120
IEEE Robotics & Automation Magazine - September 2017 - 121
IEEE Robotics & Automation Magazine - September 2017 - 122
IEEE Robotics & Automation Magazine - September 2017 - 123
IEEE Robotics & Automation Magazine - September 2017 - 124
IEEE Robotics & Automation Magazine - September 2017 - 125
IEEE Robotics & Automation Magazine - September 2017 - 126
IEEE Robotics & Automation Magazine - September 2017 - 127
IEEE Robotics & Automation Magazine - September 2017 - 128
IEEE Robotics & Automation Magazine - September 2017 - 129
IEEE Robotics & Automation Magazine - September 2017 - 130
IEEE Robotics & Automation Magazine - September 2017 - 131
IEEE Robotics & Automation Magazine - September 2017 - 132
IEEE Robotics & Automation Magazine - September 2017 - 133
IEEE Robotics & Automation Magazine - September 2017 - 134
IEEE Robotics & Automation Magazine - September 2017 - 135
IEEE Robotics & Automation Magazine - September 2017 - 136
IEEE Robotics & Automation Magazine - September 2017 - 137
IEEE Robotics & Automation Magazine - September 2017 - 138
IEEE Robotics & Automation Magazine - September 2017 - 139
IEEE Robotics & Automation Magazine - September 2017 - 140
IEEE Robotics & Automation Magazine - September 2017 - 141
IEEE Robotics & Automation Magazine - September 2017 - 142
IEEE Robotics & Automation Magazine - September 2017 - 143
IEEE Robotics & Automation Magazine - September 2017 - 144
IEEE Robotics & Automation Magazine - September 2017 - 145
IEEE Robotics & Automation Magazine - September 2017 - 146
IEEE Robotics & Automation Magazine - September 2017 - 147
IEEE Robotics & Automation Magazine - September 2017 - 148
IEEE Robotics & Automation Magazine - September 2017 - 149
IEEE Robotics & Automation Magazine - September 2017 - 150
IEEE Robotics & Automation Magazine - September 2017 - 151
IEEE Robotics & Automation Magazine - September 2017 - 152
IEEE Robotics & Automation Magazine - September 2017 - 153
IEEE Robotics & Automation Magazine - September 2017 - 154
IEEE Robotics & Automation Magazine - September 2017 - 155
IEEE Robotics & Automation Magazine - September 2017 - 156
IEEE Robotics & Automation Magazine - September 2017 - 157
IEEE Robotics & Automation Magazine - September 2017 - 158
IEEE Robotics & Automation Magazine - September 2017 - 159
IEEE Robotics & Automation Magazine - September 2017 - 160
IEEE Robotics & Automation Magazine - September 2017 - 161
IEEE Robotics & Automation Magazine - September 2017 - 162
IEEE Robotics & Automation Magazine - September 2017 - 163
IEEE Robotics & Automation Magazine - September 2017 - 164
IEEE Robotics & Automation Magazine - September 2017 - 165
IEEE Robotics & Automation Magazine - September 2017 - 166
IEEE Robotics & Automation Magazine - September 2017 - 167
IEEE Robotics & Automation Magazine - September 2017 - 168
IEEE Robotics & Automation Magazine - September 2017 - 169
IEEE Robotics & Automation Magazine - September 2017 - 170
IEEE Robotics & Automation Magazine - September 2017 - 171
IEEE Robotics & Automation Magazine - September 2017 - 172
IEEE Robotics & Automation Magazine - September 2017 - 173
IEEE Robotics & Automation Magazine - September 2017 - 174
IEEE Robotics & Automation Magazine - September 2017 - 175
IEEE Robotics & Automation Magazine - September 2017 - 176
IEEE Robotics & Automation Magazine - September 2017 - 177
IEEE Robotics & Automation Magazine - September 2017 - 178
IEEE Robotics & Automation Magazine - September 2017 - 179
IEEE Robotics & Automation Magazine - September 2017 - 180
IEEE Robotics & Automation Magazine - September 2017 - 181
IEEE Robotics & Automation Magazine - September 2017 - 182
IEEE Robotics & Automation Magazine - September 2017 - 183
IEEE Robotics & Automation Magazine - September 2017 - 184
IEEE Robotics & Automation Magazine - September 2017 - Cover3
IEEE Robotics & Automation Magazine - September 2017 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2023
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2022
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2021
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2020
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2019
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2018
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2017
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2016
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2015
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2014
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2013
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2012
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_june2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_march2011
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_december2010
https://www.nxtbook.com/nxtbooks/ieee/roboticsautomation_september2010
https://www.nxtbookmedia.com