IEEE Consumer Electronics Magazine - July/August 2022 - 54

Abandoned Object Detection Using HLDNet
Table 4. IOU Results of Proposed Method. (a) IOU Values
When the Owner is Carrying Hand Luggage. (b) IOU
Values for Five Frames in the Dumping Situation.
Video
V1
V2
V3
V4
V5
V6
V7
V8
V9
V10
V11
Scenario
Outdoor
Outdoor
Outdoor
Outdoor
Night
(a)
0.6487
0.8344
0.8446
0.8838
0.6838
Illumination change 0.6782
Illumination change 0.8479
Illumination change 0.6617
0.8682
0.8872
Indoor
Indoor
Crowded
Average
-
0.7839
(b)
0.8186
0.8903
0.8788
0.9398
0.8210
0.8315
0.8940
0.8251
0.9372
0.9529
-
0.8789
calculated based on the number of successes,
which is widely used in existing abandoned
object detection methods. Unlike conventional
abandoned object detection methods, the proposed
method determines abandoned objects
based on the hand luggage. The accurately
detected hand luggage provides a high success
rate in detecting an abandoned object. In this
context, the proposed method requires a high
IOU value since it tracks the detected hand luggage.
Table 4(a) shows the IOU values when the
owner is carrying hand luggage in each scenario.
In this experiment, the average of the IOU was
equal to 0.7839. Because IOU result in (a) is walking
after carrying hand luggage, it shows lower
results than (b) due to various factors such as
occlusion but, which is sufficient to track hand
luggage. To detect an abandoned object, an accurate
detection of dumping situation is the most
important. The histogram of the detected hand
luggage is compared to determine whether it is
the same object for five times in a row, and then
tracking is performed. Figure 10 shows the
results of detection in five consecutive frames in
a hand luggage dumping situation. It can confirm
that the proposed method shows outstanding
tracking results every frame. Table 4(b) shows
IOU values for five frames in the hand luggage
54
Conclusion
In this article, we proposed a system that
detects an abandoned object based on the
detected hand luggage using a deep neural network.
The proposed system first tracks the
pedestrian to detect the pedestrian's hand luggage.
To overcome the challenging issues in
detecting an unclassified hand luggage, we combined
a keypoint detection and object detection
networks. The abandoned object is finally
detected using the distance between the
detected hand luggage and owner. Since the proposed
system can recognize the dumping situation
based on the distance between the owner
and the hand luggage, the proposed two-step
method was verified from the experimental
results that there were fewer false detection
problems than existing single-step abandoned
object detection methods. The performance of
the proposed network was verified by measuring
the IOU of the detected hand luggage, and the
experimental results using ABODA dataset also
showed better results than existing methods.
Although the proposed method fails in a
crowded environment scenario, it can be
improved in the future research by improving
the performance of pedestrian detection.
The proposed method is a learning-based
detection method that does not use background
subtraction and can overcome the disadvantages
of using background subtraction-based detection
IEEE Consumer Electronics Magazine
Figure 10. Results of an abandoned object in five
frames.
dumping situation shown in Figure 10. The average
IOU was 0.8789, which shows the outstanding
results in the hand luggage dumping situation.
IEEE Consumer Electronics Magazine - July/August 2022

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - July/August 2022
