Computational Intelligence - May 2017 - 37

to satisfy requirements of a testing criterion. The most popular
criteria used in the FM-based testing are described next.
1. pairwise testing in the Fm Context
To derive products for the variability testing of SPLs some works
in the literature are based on combinatorial testing [30]-[33].
Pairwise testing is one of the most popular kinds of combinatorial testing, therefore, it is also applied in our work. The goal is to
generate a set of products that include all the valid pairs of features from the FM.The number of covered pairs can also be used
to evaluate the set of products that are generated. As an example,
consider again the FM in Figure 1. The pair (GPS, Basic) is
invalid, and should not be required. Considering only the variabilities, we see that the product in Figure 2 (a) includes the
pair (High Resolution, Camera) and does not include
the pair (GPS, MP3). To derive the pairs, we use the Combinatorial tool2 that implements the Automatic Efficient Test Generator (AETG) algorithm, introduced by Cohen et al. [34].
2. mutation testing in the Fm context
Mutation testing is a fault-based criterion and has been recently explored in the FM context [35]-[39]. As is the case in the
conventional mutation testing of programs [40], mutant FMs
are generated with operators that describe possible faults that
can be present in the FM. The goal is to generate a product
that is capable of distinguishing the behavior of the FM being
tested from its mutants. To this end, the product p is checked
by using an FM analyzer. The mutant is considered dead if p is
valid according to the original FM and invalid for the mutant,
or otherwise, p is invalid for the FM and valid for the mutant.
When both FMs, original and mutant ones, validate the same
set of products, they are considered equivalent. At the end, a
mutation score is calculated, given by the number of dead
mutants and total of non-equivalent generated mutants. Similar
to the pairwise coverage, the score can be used to evaluate the
adequacy of a set of products or to improve an existing one. To
illustrate this, Figure 3 shows the application of a mutation
2

http://161.67.140.42/CombTestWeb [34].

Mobile Phone
Media
Camera

Screen

Colour

Calls

High
Resolution

Calls

High
Resolution

Media

Screen

Camera

High
Resolution

(a)

(b)

Figure 2 Example of products generated from the FM in Figure 1.
(a) Valid product. (b) Invalid product.

operator in FM in Figure 1. This operator changes a requires
relation to an excludes one. In this sense, the product in
Figure  2 (a) kills the mutant, since it is valid for the original
FM and is invalid for the mutant. In our work, we used the set
of mutation operators and the FMTS tool proposed by Ferreira
et al. [36], which considers FM extensions [41] using UML-like
multiplicities. FMTS works with the framework Feature Model
Analyser (FaMa) [42], which is responsible to validate the models and execute the test data.
III. Related Work

We have observed in the last few years a growing number of
search-based approaches for different activities of SPL engineering [11], including software testing, as well as a higher
interest in the application of HHs in Search Based Software
Engineering (SBSE) [18]. This section reviews works on
both subjects.
An approach for minimization of test case sets was proposed
by Wang et al. [7]. They use a Genetic Algorithm (GA) and an
aggregation function of the following factors: the number of test
cases, pairwise coverage and capability to reveal faults. The
authors also address prioritization of test cases [8], by using
another aggregation function including cost measures, and comparing GA with (1+1) EA and random search. Other factors,
such as execution cost and resources, are also considered in
another paper of the authors [8]. Ensan et al. [4] also use a simple
GA with an aggregation function comprised of cost and error

Original FM
Screen

Mobile Phone

Mutant FM
Screen

Media
Applying
the Mutation
Operator
Camera

Colour

Calls

High
Resolution

Media

Camera

The relationship between features is changed from requires to excludes.
Figure 3 Example of a mutant generated for FM in Figure 1.

may 2017 | IEEE ComputatIonal IntEllIgEnCE magazInE

37



Table of Contents for the Digital Edition of Computational Intelligence - May 2017

Computational Intelligence - May 2017 - Cover1
Computational Intelligence - May 2017 - Cover2
Computational Intelligence - May 2017 - 1
Computational Intelligence - May 2017 - 2
Computational Intelligence - May 2017 - 3
Computational Intelligence - May 2017 - 4
Computational Intelligence - May 2017 - 5
Computational Intelligence - May 2017 - 6
Computational Intelligence - May 2017 - 7
Computational Intelligence - May 2017 - 8
Computational Intelligence - May 2017 - 9
Computational Intelligence - May 2017 - 10
Computational Intelligence - May 2017 - 11
Computational Intelligence - May 2017 - 12
Computational Intelligence - May 2017 - 13
Computational Intelligence - May 2017 - 14
Computational Intelligence - May 2017 - 15
Computational Intelligence - May 2017 - 16
Computational Intelligence - May 2017 - 17
Computational Intelligence - May 2017 - 18
Computational Intelligence - May 2017 - 19
Computational Intelligence - May 2017 - 20
Computational Intelligence - May 2017 - 21
Computational Intelligence - May 2017 - 22
Computational Intelligence - May 2017 - 23
Computational Intelligence - May 2017 - 24
Computational Intelligence - May 2017 - 25
Computational Intelligence - May 2017 - 26
Computational Intelligence - May 2017 - 27
Computational Intelligence - May 2017 - 28
Computational Intelligence - May 2017 - 29
Computational Intelligence - May 2017 - 30
Computational Intelligence - May 2017 - 31
Computational Intelligence - May 2017 - 32
Computational Intelligence - May 2017 - 33
Computational Intelligence - May 2017 - 34
Computational Intelligence - May 2017 - 35
Computational Intelligence - May 2017 - 36
Computational Intelligence - May 2017 - 37
Computational Intelligence - May 2017 - 38
Computational Intelligence - May 2017 - 39
Computational Intelligence - May 2017 - 40
Computational Intelligence - May 2017 - 41
Computational Intelligence - May 2017 - 42
Computational Intelligence - May 2017 - 43
Computational Intelligence - May 2017 - 44
Computational Intelligence - May 2017 - 45
Computational Intelligence - May 2017 - 46
Computational Intelligence - May 2017 - 47
Computational Intelligence - May 2017 - 48
Computational Intelligence - May 2017 - 49
Computational Intelligence - May 2017 - 50
Computational Intelligence - May 2017 - 51
Computational Intelligence - May 2017 - 52
Computational Intelligence - May 2017 - 53
Computational Intelligence - May 2017 - 54
Computational Intelligence - May 2017 - 55
Computational Intelligence - May 2017 - 56
Computational Intelligence - May 2017 - 57
Computational Intelligence - May 2017 - 58
Computational Intelligence - May 2017 - 59
Computational Intelligence - May 2017 - 60
Computational Intelligence - May 2017 - 61
Computational Intelligence - May 2017 - 62
Computational Intelligence - May 2017 - 63
Computational Intelligence - May 2017 - 64
Computational Intelligence - May 2017 - 65
Computational Intelligence - May 2017 - 66
Computational Intelligence - May 2017 - 67
Computational Intelligence - May 2017 - 68
Computational Intelligence - May 2017 - 69
Computational Intelligence - May 2017 - 70
Computational Intelligence - May 2017 - 71
Computational Intelligence - May 2017 - 72
Computational Intelligence - May 2017 - Cover3
Computational Intelligence - May 2017 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202311
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202308
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202305
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202302
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202211
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202208
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202205
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202202
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202111
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202108
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202105
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202102
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202011
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202008
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202005
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202002
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201911
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201908
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201905
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201902
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201811
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201808
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201805
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201802
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter12
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall12
https://www.nxtbookmedia.com