IEEE Signal Processing - July 2018 - 52

the far-infrared camera as shown in rows 14 and 15 in Figure 5.
The category baseline contained four simple videos where
Color saturation was the main challenge in this category, which
the background was always static in all of the video frames. Figdegraded the performance of majority of the compared methure 5 (rows 1 and 2) showed that most of the compared methods
ods in each category (F1 score less than 80% in Table 3). Only
in each category produced a good quality of foreground objects
for these sequences. The DB category comprised six challengone provable method (ReProCS), and eight heuristics methods
ing videos (three of them were shown in rows 3-5 in Figure 5)
(ReProCS, 3TD, 2PRPCA, PRMF, OR-PCA-illum, MSCL,
depicting outdoor scenes. This was the most difficult among
GOSUS, and COROLLA) were able to discriminate the backall categories for mounted camera object
ground-foreground pixels effectively in the
detection, which contained sequences exhibpresence of color saturation. The camera
The rPCA problem has
iting DB motions because of rippling of
jitter category comprised one indoor and
been extensively studied
water surfaces and swaying of bushes. One
three outdoor videos (rows 6 and 7 in Figduring the last seven
scheme in the provable methods category
ure 5) where the background scene underto ten years. Dynamic
(ReProCS), three in the heuristics methods
goes jitter induced motion throughout the
category (3TD, 2PRPCA, PRMF), and
video frames.
rPCA or rST has received
many in the heuristics methods with specific
significant attention
constraints category estimated a better qualComparison with nonrobust principal
much more recently, and
ity of foreground objects than all of the comcomponent analysis methods
there are many unsolved
pared methods in these categories. Table 3
We evaluated the RPCA methods against
important questions.
shows an average F1 score of close to 80%
the top methods in the CDnet 2012 data
set. We excluded the supervised methods
and more than 80% for these methods
based on deep learning because RPCA methods are unsubecause of the oversmoothing and spatiotemporal constraints
pervised ones. We selected three top performing unsuperenforced on these methods. In contrast, all other methods in
vised methods called SuBSENSE [76], PAWCS [75], and
these categories generated a noisy foreground segments because
LOBSTER [77]. For a fair comparison, we retested these
of highly DB regions. This experiment demonstrates that
methods with their source code publicly available either on
ReProCS can tolerate more background changes (larger rank
the author's respective webpages or BGSLibrary, and we
r) for a given maximum outlier fraction per column (maximum
also reported the time taken by these methods on the same
support size of any foreground image).
machine as the one used for the RPCA methods. From our
The IOM category included six videos (three were shown
comparisons, the F scores of SUBSENSE, PAWCS, and
in rows 8-10 in Figure 5), which contained ghosting artifacts
LOBSTER are a little bit lower than the ones reported in the
in the detected motion. In these sequences, the moving foreCDnet 2012 data set. This is likely because we used the same
ground objects were motionless most of the time. This is a setset of parameters for all of the categories and the videos of
ting of large outlier fractions per row. As explained previously,
the data set, while the authors of SuBSENSE, PAWCS, and
all static RPCA methods that do not exploit slow subspace
LOBSTER have optimized the set of parameters for each
change will fail for this setting. This is indeed observed in our
case. In addition, the computational time is reported for
experiment. Most compared methods categories were not able
these methods by including the training time. This is again
to handle this challenge and obtained a low F1 score (shown
done to keep comparisons with RPCA-based methods fair.
in Table 3). ReProCS and ReProCS-provable achieved the best
On average, SuBSENSE, PAWCS, and LOBSTER reached
performance among all methods (provable or not) that do not
an F-score equal to 0.79, 0.81, and 0.75, respectively, which is
exploit extra problem-specific assumptions. It had an F1 score
better than many RPCA methods. But they are also slower-
of 70% since ReProCS does exploit the subspace dynamics.
the time taken is 2.8, 1.9, and 3.7 s/frame, respectively. This
Only two methods in heuristics with additional constraints
is much slower than many of the RPCA methods that also
category (SRPCA and MSCL) were better than ReProCS,
work well (are among the top five methods), e.g., ReProCS
because they include specific heuristics to detect and remove
takes only 0.7 s/frame.
motionless frames and they use spatiotemporal regularization
The reasons for both the accuracy and slow speed are
in the low-rank background model.
that 1) SuBSENSE, PAWCS, and LOBSTER used many
The shadows category comprised six videos (some of
more color and texture features than RPCA methods; and 2)
them were shown in rows 11-13 in Figure 5) exhibiting both
they use multiple cues (use additional process to deal with
strong and faint shadows. For most of the compared methods
shadows and/or sudden illumination changes). SuBSENSE
in the provable and heuristics methods categories, these viduses an advanced texture features called spatiotemporal
eos posed a great challenge (see Table 3). Provable methods
binary features in addition to the color feature. For PAWCS,
such as RPCA-GD and ReProCS, heuristics methods such
a background dictionary is iteratively updated using a local
as PRMF, and many heuristic methods with additional condescriptor based on different low-level features. For LOBstraints achieved promising performance as compared to other
STER, it is also based on a texture feature called local binamethods. We observed that some hard shadows on the ground
ry patterns to cope with sudden lighting variations in the
were still a major limitation of the top performing algorithms.
background scenes.
The thermal category comprised five sequences captured by
52

IEEE Signal Processing Magazine

July 2018

Table of Contents for the Digital Edition of IEEE Signal Processing - July 2018