IEEE Consumer Electronics Magazine - July/August 2022 - 50

Abandoned Object Detection Using HLDNet
network. The OpenPose network used a model
that is pretrained with COCO datasets to detect
hand location. The training of the proposed
model follows the same strategy used in the
existing SSD to detect various objects. The proposed
network takes images as input to the
OpenPose network and the SSD object detection
network, respectively, as shown in Figure 4. In
the SSD object detection network, training is performed
by resizing to 300 300 size and configuring
the vgg-16 layer and 5 extra layers. The
training parameters include batch size 16, learning
rate 103, momentum 0.9, and weight decay
0.0005. We train the network for 120 000 iterations,
which takes about 21 hours on an RTX
2080Ti GPU. The training of the proposed model
follows the same strategy used in the SSD.
The loss function in the SSD is a combination
of i) the cross entropy loss, denoted as Lconf and
ii) bounding box regression loss, denoted as Lloc
Lðx; c; l; gÞ¼
1
N
ðLconfðx; cÞþ aLlocðx; l; gÞÞ:
(2)
Lconf ¼
where
c^p
and xp
expð^cp
i ¼ P
i Þ
Pos expð^cp
i Þ
In (2), Lconf is given as
X
xp
i2Pos
ijlog ð^cp
i Þ
X
i2Neg
(3)
(4)
ij means that the ith default box in a certain
grid matches the jth ground truth box in the pth
class, and c represents the confidence of multiple
classes.
In (2), Lloc is given as
Llocðx; l; gÞ¼
where
g^cx
j ¼ðgcx
g^cy
g^w
j dcx
j ¼ðgcy
j ¼ log ð
gw
dwÞ; ^gh
j
i
j dcy
i Þ=dw
i
i Þ=dh
i
j ¼ log ð
gh
dh
j
i
Þ
X X
i2Posm2fcx;cy;w;hg
(5)
(6)
(7)
(8)
where SL1 is the robust bounding box regression
loss using '1 smoothing, which is known to be
efficient to predict a rectangular region using
50
xk
ijSL1ðlm
i ^gm
j Þ
log ð^c0
i Þ
regression. l represents the predicted box,
ðcx; cyÞ the center coordinate, d the default
bounding box, fw; hg the width and height of the
bounding box, ^gcx
j and ^gcy
with respect to cx, and cy, ^gw
j are normalized boxes
j , and ^gh
j are normalized
boxes with respect to the width and height.
of the default box. We modified the pretrained
VGG-16 as a feature extractor. Six extra layers
minimize two loss functions: a confidence loss
using softmax in (2) and an L1 loss that calculates
the offset of the bounding box. In SSD, a
default anchor box in Faster R-CNN was used
sk ¼ smin þ
smax smin
m 1
ðk 1Þ;k 2½1;m
ar 2f1; 2; 3;
wa
k ¼ sk
p
ffiffiffiffiffi
ar
;ha
1
2
; g
1
3
k ¼ sk=
p
ffiffiffiffiffi
ar
(9)
(10)
(11)
where S represents the scale of the box, k an
integer from 1 to 6, and ar a prespecified aspect
ratio. After determining the offset of the default
box using (9), the IOU is compared with each
grid cell of the feature map and the GT, and the
learning proceeds by selecting those with a
threshold value of 0.5 or more. We selected only
bounding boxes with high confidence loss to
maintain the ratio of positive and negative as
1:3. Based on the reliability between the selected
default box and the abandoned object class, the
confidence loss of (3) is obtained. Bounding box
regression is performed using the final confidence
score and bounding box coordinates.
After learning, the objects with overlapping offsets
are removed through the NMS operation,
and the reliability of detected objects far from
the hand position is adjusted through Gaussian
filtering to detect the final hand luggage. Figure 6
shows the training curve of the proposed
network.
Abandoned Object Detection System
In this section, we present an abandoned
object decision system using the previously
detected hand luggage region in section " HLD
Network. " Figure 7 shows the framework of the
proposed abandoned object decision system.
SSD can detect only luggage in hand, but it cannot
detect an abandoned object. The proposed
method combined the SSD detector with KCF
IEEE Consumer Electronics Magazine
IEEE Consumer Electronics Magazine - July/August 2022

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - July/August 2022
