Signal Processing - January 2017 - 99

where the within-class scatter matrix
S W is defined as
SW =

L

N,

/ / [x (j,) - nv ,] [(x (j,) - nv ,] T .

,=1 j=1

(22)

The between-class scatter matrix S B is
defined as
SB =

L

/

,=1

N , 6nv , - nv@6nv , - nv@ T (23)

where N , denotes the number of training vectors associated with the ,th
class, nv , denotes the centroid of the , th
class, for l = 1, f , L , and L denotes
the number of different classes. The
between-class scatter matrix represents
the so-called signal subspace as it is
formed from the L class-discriminating
vectors learnable from the data set.
To facilitate the training of Ω from
the supervised data set, we adopt two
coordinate transformations to simplify
the mathematical analysis.
■■ Orthogonalization Transforma-
tion on U. Without loss of generality, we assume that U is already
pre-orthogonalized in (8), which can
then be expressed as
u u Rx
Ω = R x UU
T

(24)

u !0
where U
with column vectors
u i = v -u i1 U i, i = 1, f, n.
defined as U
Whitening
Transformation on x.
1
Let R x2 denote the square-root of
the covariance
matrix R x i.e.,
1
T
2
2
R x = R x R x . By transforming the
original vector space to a "canonical"
(or "whitened") vector space via
M#n

■■

-1

xl = R x 2 x ,

(25)

the new covariance matrix becomes
an identity matrix, i.e., R xl = I. It
follows that
t l = R xl U
u lU
u l T R xl = U
u lU
u l T,
X

T

3 4 5

4
3
2
1
0
-1
-2
-3
-4
-5 -4 -3 -2 -1 0 1 2
x (1)
(b)

3 4 5

Figure 5. A display of DUCA-reduced covariance matrices for the DIP example, where x = 6x (1) x (2)@T .
(a) PP query, with the setting a = 1 and b = 1, and (b) privacy-intrusive query, with a = 0 and
b = - 1.

In the whitened space, the betweenclass utility scatter matrix SlB U points to
the L best utility-class-discriminating
vectors, spanning an (L - 1) -dimensional subspace. Therefore, pursuant to
(26), it is natural to associate each colu l with a vector pointing from
umn of U
the mass center to the centroid of a utility class. This results in

1

T

Eq. 24

T

= R x2 Xl R x2 .

Eq. 26

(29)

It follows that
T
t lR 2 = X
t
X
x Eq. 29 .

(30)

Utility-driven DIM-DCA
for supervised learning

Algorithm 1. Utility-driven DCA
learning algorithm
The optimization formulation of DIMDCA involves searching for the projection matrix W DCA ! 0 M # m:
T

W DCA =

(27)

(Note that t here assumes the role of
the variance v 2 .) The optimal DCA

argmax

IEEE Signal Processing Magazine

(32)

mi =

v Ti

v Ti S BU v i
if i = 1, f, L - 1
(Sr + tI) v i
0 if i $ L .
(33)
4

Metric for interclass separability

Now we are ready to establish a machine
learning variant, called DIM-DCA, corresponding to (7).

{W: W T 6Sr + tI@W = I}

Equivalently, they can be derived from
the first m principal eigenvectors of

*

T
1
SlB U R x2 ? R x2
Eq. 28

(31)

The extracted queries are rankordered according to their "signal to
power ratios," which are equivalent to
their corresponding eigenvalues:

u u T R x = R x2 U
u lU
u l T R x2
X ? R x UU
Eq. 27

1
S B U = R x2
Eq. 25

-1
6Sr + tI@ S B U .

(28)

Note also that

1

solution can be derived from the principal eigen-subspace of the following
"discriminant matrix" [6]:

eig ^ S B U, Sr + tIh .

t l.
u lU
u lT = X
SlB U ? U
Eq. 26

(26)

where
u l / R x2 U
u.
U

x (2)

Sr = S B + S W ,

4
3
2
1
0
-1
-2
-3
-4
-5 -4 -3 -2 -1 0 1 2
x (1)
(a)

x (2)

In supervised learning, the scatter
matrix Sr can be further divided into two
additive parts [5]

tr ^W S B U W h.

|

January 2017

|

The trace-norm of the discriminant matrix,
defined in (31), may be used as a simple
metric to measure the inter-class separability of a supervised data set. It offers a
convenient tool to evaluate the the suitability of a certain similarity function (or
kernel function [3]) to be chosen for nonlinear data analysis (see the next section).

Theoretical connection
between two variants of DCA
In [6], another variant of DCA was
developed for finding the optimal subspace projection matrix via the principal
eigenvectors of
99



Table of Contents for the Digital Edition of Signal Processing - January 2017

Signal Processing - January 2017 - Cover1
Signal Processing - January 2017 - Cover2
Signal Processing - January 2017 - 1
Signal Processing - January 2017 - 2
Signal Processing - January 2017 - 3
Signal Processing - January 2017 - 4
Signal Processing - January 2017 - 5
Signal Processing - January 2017 - 6
Signal Processing - January 2017 - 7
Signal Processing - January 2017 - 8
Signal Processing - January 2017 - 9
Signal Processing - January 2017 - 10
Signal Processing - January 2017 - 11
Signal Processing - January 2017 - 12
Signal Processing - January 2017 - 13
Signal Processing - January 2017 - 14
Signal Processing - January 2017 - 15
Signal Processing - January 2017 - 16
Signal Processing - January 2017 - 17
Signal Processing - January 2017 - 18
Signal Processing - January 2017 - 19
Signal Processing - January 2017 - 20
Signal Processing - January 2017 - 21
Signal Processing - January 2017 - 22
Signal Processing - January 2017 - 23
Signal Processing - January 2017 - 24
Signal Processing - January 2017 - 25
Signal Processing - January 2017 - 26
Signal Processing - January 2017 - 27
Signal Processing - January 2017 - 28
Signal Processing - January 2017 - 29
Signal Processing - January 2017 - 30
Signal Processing - January 2017 - 31
Signal Processing - January 2017 - 32
Signal Processing - January 2017 - 33
Signal Processing - January 2017 - 34
Signal Processing - January 2017 - 35
Signal Processing - January 2017 - 36
Signal Processing - January 2017 - 37
Signal Processing - January 2017 - 38
Signal Processing - January 2017 - 39
Signal Processing - January 2017 - 40
Signal Processing - January 2017 - 41
Signal Processing - January 2017 - 42
Signal Processing - January 2017 - 43
Signal Processing - January 2017 - 44
Signal Processing - January 2017 - 45
Signal Processing - January 2017 - 46
Signal Processing - January 2017 - 47
Signal Processing - January 2017 - 48
Signal Processing - January 2017 - 49
Signal Processing - January 2017 - 50
Signal Processing - January 2017 - 51
Signal Processing - January 2017 - 52
Signal Processing - January 2017 - 53
Signal Processing - January 2017 - 54
Signal Processing - January 2017 - 55
Signal Processing - January 2017 - 56
Signal Processing - January 2017 - 57
Signal Processing - January 2017 - 58
Signal Processing - January 2017 - 59
Signal Processing - January 2017 - 60
Signal Processing - January 2017 - 61
Signal Processing - January 2017 - 62
Signal Processing - January 2017 - 63
Signal Processing - January 2017 - 64
Signal Processing - January 2017 - 65
Signal Processing - January 2017 - 66
Signal Processing - January 2017 - 67
Signal Processing - January 2017 - 68
Signal Processing - January 2017 - 69
Signal Processing - January 2017 - 70
Signal Processing - January 2017 - 71
Signal Processing - January 2017 - 72
Signal Processing - January 2017 - 73
Signal Processing - January 2017 - 74
Signal Processing - January 2017 - 75
Signal Processing - January 2017 - 76
Signal Processing - January 2017 - 77
Signal Processing - January 2017 - 78
Signal Processing - January 2017 - 79
Signal Processing - January 2017 - 80
Signal Processing - January 2017 - 81
Signal Processing - January 2017 - 82
Signal Processing - January 2017 - 83
Signal Processing - January 2017 - 84
Signal Processing - January 2017 - 85
Signal Processing - January 2017 - 86
Signal Processing - January 2017 - 87
Signal Processing - January 2017 - 88
Signal Processing - January 2017 - 89
Signal Processing - January 2017 - 90
Signal Processing - January 2017 - 91
Signal Processing - January 2017 - 92
Signal Processing - January 2017 - 93
Signal Processing - January 2017 - 94
Signal Processing - January 2017 - 95
Signal Processing - January 2017 - 96
Signal Processing - January 2017 - 97
Signal Processing - January 2017 - 98
Signal Processing - January 2017 - 99
Signal Processing - January 2017 - 100
Signal Processing - January 2017 - 101
Signal Processing - January 2017 - 102
Signal Processing - January 2017 - 103
Signal Processing - January 2017 - 104
Signal Processing - January 2017 - 105
Signal Processing - January 2017 - 106
Signal Processing - January 2017 - 107
Signal Processing - January 2017 - 108
Signal Processing - January 2017 - 109
Signal Processing - January 2017 - 110
Signal Processing - January 2017 - 111
Signal Processing - January 2017 - 112
Signal Processing - January 2017 - 113
Signal Processing - January 2017 - 114
Signal Processing - January 2017 - 115
Signal Processing - January 2017 - 116
Signal Processing - January 2017 - Cover3
Signal Processing - January 2017 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201809
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201807
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201805
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201803
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201801
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1117
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0917
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0717
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0517
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0317
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0117
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1116
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0916
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0716
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0516
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0316
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0116
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1115
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0915
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0715
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0515
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0315
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0115
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1114
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0914
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0714
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0514
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0314
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0114
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1113
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0913
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0713
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0513
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0313
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0113
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1112
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0912
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0712
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0512
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0312
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0112
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1111
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0911
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0711
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0511
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0311
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0111
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1110
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0910
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0710
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0510
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0310
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0110
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1109
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0909
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0709
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0509
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0309
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0109
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1108
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0908
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0708
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0508
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0308
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0108
https://www.nxtbookmedia.com