IEEE Consumer Electronics Magazine - July/August 2022 - 46

Abandoned Object Detection Using HLDNet
Table 1. Various Studies on Abandoned Object Detection
Based on Background Subtraction.
Method
Background subtraction-based
Robust background subtraction-based
Background subtraction + Deep learningbased
Reference
3-5
6-8
8-10
approaches
to the abandoned object detection
problem. Although violence and slipping behaviors
can be detected by itself, detection of abandonment
requires a series of processes to train a
dumping action and to determine whether the
object is intentionally abandoned in a sufficient
number of image frames. In this context, a deep
learning approach can also classify the abandoned
object.
We define that an abandoned object is
smart village.1;2 Among various abnormal behaviors,
abandoned object detection refers to
decide if a pedestrian intentionally dumps an
object at a certain place. Accurate analysis of
abandonment behavior is required to determine
crucial situations such as illegal garbage dumping
or installing explosives in a public place.
Abandonment behavior is determined based
on detection of an abandoned object. In existing
abandoned object detection methods, an object
is determined as an abandoned one if it stays at
a fixed position for a prespecified time duration.
For that reason, most abandoned object detection
algorithms use a background subtraction
method to detect the static object. Among them,
the dual background subtraction is an improved
method that uses two pieces of background
information with different updating periods. It
can better detect static objects than a simple
background subtraction method since slowly
changing foreground information is absorbed in
the background. However, the dual method is
still vulnerable to various challenging factors
affecting the background, such as illumination
change or camera shaking. Therefore, the detection
method depending on the background information
is not suitable for a pan-tilt-zoom (PTZ)
camera or dynamic outdoor environment.
Most of the abandoned object detection
methods used the background subtraction
method, which removes a static background by
detecting the difference between two adjacent
video frames and using background modeling. In
the field of abandoned object detection, it is
known as a simple, efficient method that can
extract moving objects from fixed background.
Table 1 refers to studies of abandoned object
detection based on various background subtraction.
Unlike other abnormal behaviors such as
violence or slip, there were few deep learning
46
dumped by a human hand. We added the condition
since it is difficult to recognize the dumping
situation using a simple background subtraction
method. The background subtraction-based
object detection methods has a fundamental
problem of false detection due to dynamic background
such as flickering neon signs and static
objects without motion. To prevent such false
detection, we define an abandoned object as an
object dumped by a human. In this context, to
detect an abandoned object after recognizing
the dumping situation, the abandoned object
should be detected by tracking the object in the
hand. Although it is difficult to detect an unspecified
abandoned object using a learning-based
method, but it is possible to detect an object carried
by a human hand. Therefore, we propose
first locate the object located carried by the
hand. Next, we classify it as a hand luggage class.
We then determine abandonment behavior by
detecting the hand luggage. More specifically,
the unspecified hand luggage is detected using a
deep neural network that merges keypoint and
objects detection. To track the hand luggage, we
propose a learning-based detection method that
is robust to challenging factors, such as illumination
changes and camera shaking.
The article is organized as follows. Section
" Proposed Method " presents the proposed
abandonment behavior detection algorithm
based on the HLD. Section " Experimental
Results " summarizes experimental results, and
Section " Conclusion " concludes the article.
Proposed Method
This section describes the abandoned object
detection system using learning-based hand luggage
localization. The proposed system consists
of three parts as shown in Figure 1: i) pedestrian
IEEE Consumer Electronics Magazine
IEEE Consumer Electronics Magazine - July/August 2022

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - July/August 2022
