IEEE Geoscience and Remote Sensing Magazine - December 2020 - 127

second annotator for tune-ups. Finally, the third annotator screened all generated 5-s videos and removed videos
with quite similar contents. Moreover, they were asked to
double-check whether these videos were taken by UAVs.
Overall, the total annotation time was around 290 h.

intraclass variation and interclass similarity and captures
dynamic events in various circumstances and at dramatically various scales.
COLLECTION AND ANNOTATION
We start by creating our taxonomy (compare Figure 3) by
building a set of the 24 most commonly seen events from
aerial scenes. Moreover, to investigate whether models can
distinguish events from normal videos, we set up a category called nonevent, which comprises videos not including
specific events. The 25 classes of our data set can be found
in Figure 2.
To collect candidate videos for further labeling, we
searched YouTube by parsing the metadata of videos and
crawling the search engine to create a collection of candidate videos for each category. As a result, we collected
1,120 long videos. Note that we set drone and UAV as keywords to exclude videos not taken from UAVs. Then we
downloaded all videos and sent them to data annotators.
Each annotator was asked to localize 5-s video snippets
that depict specific events and cut them from the long
candidate videos.
Three different paid annotators were responsible for validating cut videos during the annotation procedure. The first
annotator generated primary annotations to begin with.
Then, annotations from the first round were sent to the

Baseball

DATA SET STATISTICS
The goal of this article is to collect a large, diverse data set
that can be used to train models for event recognition in
UAV videos. As we gather aerial videos from YouTube, the
largest video sharing platform in the world, we are able to
include a large breadth of diversity, which is more challenging than making use of self-collected data [4], [5].
In total, we gathered and annotated 2,864 videos for 25
classes. Each video sequence is at 24 frames per second,
for 5 s, and with a spatial size of 640 × 640 pixels. The
train/test split can be found in Figure 4 and in the " Experimental Setup " section. Figure 4 exhibits the distribution
of all classes. The red and blue bars represent the numbers
of training and test samples in each category, respectively,
and the green bars denote the total number of instances in
each category.
To build a diverse data set, we collect not only highquality UAV videos but also ones acquired in extreme conditions. By doing so, many challenging issues of event recognition in overhead videos in real-world scenarios, e.g., low

Police Chase

Security

Conflict

Basketball
Boating

Sport

Post-Earthquake
Flood

Disaster

Cycling

Fire

Running

Landslide

Soccer

Mudslide

Swimming
Car Racing
ERA Data Set

Traffic

Traffic Collision
Traffic Congestion

Harvesting

Party
Concert
Parade/Protest
Religious Activity

Ploughing
Social Activity

Productive Activity

Constructing

Nonevent

FIGURE 3. The categorization of event classes in the ERA data set. All event categories are arranged in a two-level tree, with 25 leaf nodes
connected to seven nodes at the first level, i.e., disaster, traffic, productive activity, security, sport, social activity, and nonevent.
DECEMBER 2020

IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE

127

IEEE Geoscience and Remote Sensing Magazine - December 2020

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - December 2020