Computational Intelligence - August 2015 - 55

where
2

H j = / T ij

(7)

i =1

This means that our ENN method
makes the prediction based on which
decision resulting the largest intra-class
coherence when we iteratively assume the
test sample Z to be each possible class.
We now present the detailed calculation steps for a two-class classification
problem using our ENN method. First,
let's assume that the observation Z should
be classified as class 1.Then, for each training sample, we re-calculate its k nearest
neighbors and obtain the new generalized
class-wise statistic according to Eq. (4),
denoted as T 11 for class 1 and T 12 for class
2. Second, we assume that the observation
Z should be classified as class 2; then, for
each training sample, we also re-calculate
its k nearest neighbors and obtain the new
generalized class-wise statistic according to
Eq. (4), denoted as T 21 for class 1 and T 22
for class 2. Then a classification decision is
made according to Eq. (6).
The aforementioned two-class ENN
decision rule can be easily extended to
multi-class classification problems. Specifically, for an N-class classification problem,
we have an ENN Classifier algorithm
(see the box below).
B. ENN.V1 Classifier: An Equivalent
Version of ENN

The underlying idea of the ENN rule
shown in Eq. (8) is that we classify the
test sample Z based on which decision
results in the greatest intra-class coherence among all possible classes. We can
further demonstrate that the classification
of Z in our ENN rule depends not only

ENN Classifier: Given an unknown
sample Z to be classified, we iteratively assign it to each possible class
j, j = 1, 2, g, N, and compute the
j
generalized class-wise statistic T i for
each class i, i = 1, 2, g, N. Then, the
sample Z is classified according to:
N

fENN = arg max / T i
j ! 1, 2, g, N i = 1

j

(8)

on who are the nearest neighbors of Z ,
but also on who consider Z as one of
their nearest neighbors.
For the new observation Z , let's assume that there are k 1 nearest neighbors
from class 1 and k 2 nearest neighbors
from class 2, where k 1 + k 2 = k is the
total number of nearest neighbors investigated. First, let's assume the observation
Z to be classified as class 1; then, we
count the number of class 1 data who
have an increased number of class 1 samples in its k nearest neighbors (denote
this number as Dn 11 ), and we also count
the number of class 2 data who have a
decreased number of class 2 samples in its
k nearest neighbors (denote this number
as Dn 12 ). Then, in a similar way, we further assume the observation Z to be a
member of class 2, and count the number of class 1 data who have a decreased
number of class 1 samples in its k nearest
neighbors (denote this number as Dn 21 ),
and we also count the number of class 2
data who have an increased number of
class 2 samples in its k nearest neighbors
(denote this number as Dn 22 ). In this way,
Dn ij represents how many of the k nearest neighbors from each class will change
because of the introduction of the new
sample Z , when it is iteratively assumed
to be each possible class. To clearly demonstrate this concept, we present a detailed calculation example in our
Supplementary Material Section 1 for a
two-class classification problem.
With these discussions, we present an
equivalent version of the ENN classifier
in the box ENN.V1 (see below).
Proof of ENN.V1. We iteratively
assign the unknown sample Z to class j
to obtain new generalized class-wise sta-

tistic T ij for class i . According to Eq. (4),
we have,
when i = j
j
Ti = 1
/
nli k X ! Si , {Z}
k

/ I r (X, Sl=S 1 , S 2 , {Z})

r =1

1
=
/
(n i + 1) k ;X ! Si
k

/ I r (X, S 1 , S 2 , {Z})

r =1

+ / I r (Z, S 1 , S 2 , {Z})G
k

r =1

1
=
/
(n i + 1) k ;X ! Si
k

/ I r (X, S 1 , S 2) + Dn ij

r =1

+ / I r (Z, S 1 , S 2)G
k

r =1

1
^ n kT + Dn ij + k i h
=
(n i + 1 ) k i i

(10)

and when i ! j
k

T ij = 1 / / I r (X, S l =S 1 , S 2 ,{Z})
nli k X ! Si r = 1
= 1 = / / I r (X, S 1 , S 2) - Dn ijG
n i k X ! Si r = 1
k

= 1 ^n i kTi - Dn ij h
ni k

Dn ij
(11)
ni k
Therefore, from Eq. (8), we have:

= Ti -

N

fENN = arg max / T ij
j ! 1, 2, g, N i = 1

= arg max / ^T ij - Ti h
N

j ! 1, 2, g, N i = 1

= arg max )^T ij - Ti hi = j
j ! 1, 2, g, N

+ / ^T ij - Ti h3
N

i!j

ENN.V1: Given an unknown sample Z to be classified, we iteratively assign it to each
possible class j, j = 1, 2, g, N, and predict the class membership according to:
fENN.V1 = arg max )e
j ! 1, 2, g, N

N
Dn i + k i - kTi o
Dn i
3
-/
(n i + 1) k i = j i ! j n i k
j

j

(9)

where k is the user-defined parameter of the number of the nearest neighbors, n i is
the number of training data for class i, k i is the number of the nearest neighbors of
j
the test sample Z from class i, Dn i represents the change of the k nearest neighbors for class i when the test sample Z is assumed to be class j, and Ti represents
the generalized class-wise statistic of original class i (i.e., without the introduction of
the test sample Z ).

August 2015 | IEEE ComputAtIonAl IntEllIgEnCE mAgAzInE

55



Table of Contents for the Digital Edition of Computational Intelligence - August 2015

Computational Intelligence - August 2015 - Cover1
Computational Intelligence - August 2015 - Cover2
Computational Intelligence - August 2015 - 1
Computational Intelligence - August 2015 - 2
Computational Intelligence - August 2015 - 3
Computational Intelligence - August 2015 - 4
Computational Intelligence - August 2015 - 5
Computational Intelligence - August 2015 - 6
Computational Intelligence - August 2015 - 7
Computational Intelligence - August 2015 - 8
Computational Intelligence - August 2015 - 9
Computational Intelligence - August 2015 - 10
Computational Intelligence - August 2015 - 11
Computational Intelligence - August 2015 - 12
Computational Intelligence - August 2015 - 13
Computational Intelligence - August 2015 - 14
Computational Intelligence - August 2015 - 15
Computational Intelligence - August 2015 - 16
Computational Intelligence - August 2015 - 17
Computational Intelligence - August 2015 - 18
Computational Intelligence - August 2015 - 19
Computational Intelligence - August 2015 - 20
Computational Intelligence - August 2015 - 21
Computational Intelligence - August 2015 - 22
Computational Intelligence - August 2015 - 23
Computational Intelligence - August 2015 - 24
Computational Intelligence - August 2015 - 25
Computational Intelligence - August 2015 - 26
Computational Intelligence - August 2015 - 27
Computational Intelligence - August 2015 - 28
Computational Intelligence - August 2015 - 29
Computational Intelligence - August 2015 - 30
Computational Intelligence - August 2015 - 31
Computational Intelligence - August 2015 - 32
Computational Intelligence - August 2015 - 33
Computational Intelligence - August 2015 - 34
Computational Intelligence - August 2015 - 35
Computational Intelligence - August 2015 - 36
Computational Intelligence - August 2015 - 37
Computational Intelligence - August 2015 - 38
Computational Intelligence - August 2015 - 39
Computational Intelligence - August 2015 - 40
Computational Intelligence - August 2015 - 41
Computational Intelligence - August 2015 - 42
Computational Intelligence - August 2015 - 43
Computational Intelligence - August 2015 - 44
Computational Intelligence - August 2015 - 45
Computational Intelligence - August 2015 - 46
Computational Intelligence - August 2015 - 47
Computational Intelligence - August 2015 - 48
Computational Intelligence - August 2015 - 49
Computational Intelligence - August 2015 - 50
Computational Intelligence - August 2015 - 51
Computational Intelligence - August 2015 - 52
Computational Intelligence - August 2015 - 53
Computational Intelligence - August 2015 - 54
Computational Intelligence - August 2015 - 55
Computational Intelligence - August 2015 - 56
Computational Intelligence - August 2015 - 57
Computational Intelligence - August 2015 - 58
Computational Intelligence - August 2015 - 59
Computational Intelligence - August 2015 - 60
Computational Intelligence - August 2015 - 61
Computational Intelligence - August 2015 - 62
Computational Intelligence - August 2015 - 63
Computational Intelligence - August 2015 - 64
Computational Intelligence - August 2015 - 65
Computational Intelligence - August 2015 - 66
Computational Intelligence - August 2015 - 67
Computational Intelligence - August 2015 - 68
Computational Intelligence - August 2015 - 69
Computational Intelligence - August 2015 - 70
Computational Intelligence - August 2015 - 71
Computational Intelligence - August 2015 - 72
Computational Intelligence - August 2015 - 73
Computational Intelligence - August 2015 - 74
Computational Intelligence - August 2015 - 75
Computational Intelligence - August 2015 - 76
Computational Intelligence - August 2015 - 77
Computational Intelligence - August 2015 - 78
Computational Intelligence - August 2015 - 79
Computational Intelligence - August 2015 - 80
Computational Intelligence - August 2015 - Cover3
Computational Intelligence - August 2015 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202311
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202308
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202305
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202302
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202211
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202208
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202205
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202202
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202111
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202108
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202105
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202102
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202011
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202008
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202005
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202002
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201911
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201908
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201905
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201902
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201811
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201808
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201805
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201802
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter12
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall12
https://www.nxtbookmedia.com