Signal Processing - November 2017 - 46

In this article, we employ the PASCAL VOC 2012
benchmark for evaluation since it has the largest record of
reported performance, thus allowing fair and comprehensive
-comparisons among various methods. The evaluation metric
is mean inter-section-over-union (mIoU) between groundtruth and predicted segmentation results. We present scores
reported in the original papers, for both validation and test
splits. For an approach with multiple variations in model
architecture, only the score of the best model is presented for
the sake of brevity. For an approach adopting various types
of weak supervision (e.g., [4], [35], [36], and [38]), we present multiple scores corresponding to the individual supervision types. In addition to annotations for training, we also
report the type of extra information adopted by each method
if exists, as such information may introduce additional supervision that are not available from the PASCAL VOC 2012
training data.
Table 1 summarizes comparison results for the weakly
supervised approaches. As discussed in the section "Weakly
Supervised Semantic Segmentation," they are categorized by

the type of supervision employed for training. Note that it is
not appropriate to compare different types of weak annotation
directly since the models employed in each method have different configurations and capacities. However, the general performance trend across various approaches given in the table
clearly demonstrates the impact of supervision levels and benefits of using extra information.
Approaches based only on image-level class labels perform poorly in general, as shown in Table 1. As described
in the section "Weakly Supervised Semantic Segmentation,"
it is mainly because the discriminative learning objective
employed in weakly supervised approaches tends to focus on
small discriminative parts. The performance is improved by
adopting additional cues such as discriminative localization
and underlying low-level image structures, since they provide
useful information to regularize the prediction during training. Also, it is clearly observed that prior knowledge such as
objectness generally improves the performance, as it helps
to estimate a better object extent by injecting class-agnostic
objectness likelihood in a pixel level. Increasing the strength

Table 1. The comparison results of weakly supervised semantic segmentation algorithms on the PASCAL VOC 2012 data set.
Supervision

Method

Extra Information

mIoU (val)

mIoU (test)

Image-level label

MIL-FCN [37]

-

25.1

25.7

WSSL [35]

-

38.2

39.6

Prior knowledge

CCNN [36]

-

35.3

36.4

AugFeed [40]

-

52.7

52.6

WTP [4]

-

29.8

-

†

MIL-SP [38]

Superpixel [12]

36.6

35.8

SPN [24]

Superpixel [60]

50.3

46.9

SEC [22]

Localization [58]†

50.7

51.7

DCSM [46]

Localization [47]†

44.1

45.1

CCNN [36]

Object size

45.1

42.4

MIL-SP [38]

Objectness [2]

42.6

40.6

AugFeed [40]

Objectness [2]

54.3

55.5

STC [54]

Objectness [2]

49.8

51.2

Saliency [34]

Saliency

55.7

56.7

Point supervision

WTP [4]

Point + objectness [1]

43.8

-

Bounding box

WSSL [35]

Bounding box

60.6

62.2

Boxsup [9]

Bounding box + objectness [2]

62.0

64.6

SDI [20]

Bounding box + objectness [2]

65.7

67.5

Scribble

ScribbleSup [26]

Scribble

63.1

-

Microannotation

CheckMask [44]

User feedback

51.5

52.9

MicroAnno [21]

User feedback

51.9

53.2

TransferNet [17]

Exclusive segmentation mask [29]

52.1

51.2

MCNN [49]

Web videos [39]

38.1

39.8

CrawlSeg [18]

Web videos (YouTube)

58.1

58.7

Additional data

†

Indicates extra information obtained without additional supervision.

46

IEEE SIGNAL PROCESSING MAGAZINE

|

November 2017

|
Table of Contents for the Digital Edition of Signal Processing - November 2017
