Systems, Man & Cybernetics - April 2017 - 27

Note that (3) holds as long as A IJ
where U (n) and V (n) are matrices for
has rank two or less. For example, if a
the left and right singular vectors,
The concept of
column of A IJ is obtained by scaling
respectively, and R (n) is a diagonal
coclustering was
matrix containing singular values.
another column and adding a con-
For n = 1, 2, ..., N, we only need to
stant, as a combination of additive and
introduced in the
multiplicative cocluster patterns, A IJ
compute U (n) and not V (n) to obtain
1970s, but it has
has rank two, and (3) is also valid. In
(5). The decomposition of a three-
general, HDSVS can be used to extract
mode tensor is illustrated in Figure 4.
only gained much
low-rank patterns embedded in a large
By detecting hyperplanes in the
attention recently
matrix or tensor.
singular-vector space with U (n), we
due to its useful
If the data have noise, we need to
can then find indices sets along
filter out small singular values. If there
mode n for cocluster identification.
applications to
is more than one cocluster in the data,
An example is shown in Figure 5.
genomic data
then the rank of the matrix can be
The index sets along three modes
larger than two, and we need to retain
are I, J, and K for 3-D data, and they
analysis.
more singular vectors. In this case,
should be understood as I (1), I (2), and
each cocluster will correspond to a
I (3) in a general notation. Assume
hyperplane in general in the singular-
that an index set along mode n
vector spaces. When coclusters are embedded in a large
is I (n); then, the scoring function in (4) can be extended to
matrix, rows and columns in the coclusters will be found from
higher dimensional coclusters:
the detected hyperplanes, while irrelevant elements will be dis-
tributed randomly off the hyperplanes. Different scenarios
S (I (1), I (2), g, I (N))
have been discussed in [25].
^ S i I gI , S I i I gI , g, S I I gi h . (7)
=
min
i ! I , i ! I , g, i ! I
Let us assume that we have found row indices sets I 1 and
I 2 from the hyperplanes detected from left singular vectors
and column indices sets J 1 and J 2 from the hyperplanes
Examples
detected from right singular vectors. There are four combina-
As an example, we show an eight-by-five matrix in Figure 6(a).
tions of these index sets: I 1 # J 1, I 1 # J 2,I 2 # J 1, and I 2 # J 2 . It
The highlighted elements in the matrix form a general linear
cocluster, which is a combination of additive and multiplicative
is possible that some of these pairs do not correspond to
coclusters. We can use the following scoring function to select
coclusters [25]:
1

^ S Ij, S iJ h
S (I, J) = imin
! I, j ! J

1
=imin
;1 - J - 1 | t ^a Ij, a Iq h,
! I, j ! J
q ! j, q ! J
1
| t^a iJ, a qJhE,
1I - 1 q ! i,q ! I

2

(4)

N

A = B # 1 U (1) # 2 U (2) # 3 g # N U (N),

(2)

N

(N)

1

M1

(2)

(N)

(1)

m1
M2

2

(3)

(N)

U1

U2

M2
m3

B
A

N

m2

m3
m1 m2

M1

(1) (2)

U3

M3

Figure 4. the decomposition of a three-mode tensor
using HOSVD. U 1 can be found by unfolding A to an
M 1 # M 2 M 3 matrix and computing the left singular
vectors of this matrix. U 2 and U 3 can be computed in
similar ways.

(5)

where B is the core tensor, U (n) (n = 1, 2, ..., N ) contains
singular vectors, and # n stands for tensor and matrix mul-
tiplication along mode n. Then, U (n) can be computed from
the mode-n unfolded matrix A (n) of the tensor. In general,
A (n) has the size of M n # M 1 g M n - 1 M n + 1 g M N and is
obtained by keeping mode n as one dimension and flatten-
ing all other modes to another dimension. The SVD of A (n)
has the following form:
A (n) = U (n) R (n) ^ V (n) hT ,

2

M3

where t represents Pearson's correlation coefficient between
two vectors. We consider that a cocluster is identified if S(I, J)
is less or equal to a prespecified value.
The HDSVS method can be extended to higher-order ten-
sors [25]. The higher-order SVD (HOSVD) [29]-[32] of an
N-mode tensor A ! R M # M # g # M is defined as
1

(1)

(6)

AIJK

Tensor A
Size
M1×M2×M3

U1
|I|
Rows

U2
|J|
Rows

U3
|K|
Rows

Figure 5. the detection of coclusters in singular

vector spaces. In this diagram, A IJK is a cocluster
embedded in a large tensor A. the index sets
I, J, and K can be determined by searching for
hyperplanes based on U 1, U 2, and U 3, respectively.

Ap ri l 2017

IEEE SyStEmS, man, & CybErnEtICS magazInE

27



Table of Contents for the Digital Edition of Systems, Man & Cybernetics - April 2017

Systems, Man & Cybernetics - April 2017 - Cover1
Systems, Man & Cybernetics - April 2017 - Cover2
Systems, Man & Cybernetics - April 2017 - 1
Systems, Man & Cybernetics - April 2017 - 2
Systems, Man & Cybernetics - April 2017 - 3
Systems, Man & Cybernetics - April 2017 - 4
Systems, Man & Cybernetics - April 2017 - 5
Systems, Man & Cybernetics - April 2017 - 6
Systems, Man & Cybernetics - April 2017 - 7
Systems, Man & Cybernetics - April 2017 - 8
Systems, Man & Cybernetics - April 2017 - 9
Systems, Man & Cybernetics - April 2017 - 10
Systems, Man & Cybernetics - April 2017 - 11
Systems, Man & Cybernetics - April 2017 - 12
Systems, Man & Cybernetics - April 2017 - 13
Systems, Man & Cybernetics - April 2017 - 14
Systems, Man & Cybernetics - April 2017 - 15
Systems, Man & Cybernetics - April 2017 - 16
Systems, Man & Cybernetics - April 2017 - 17
Systems, Man & Cybernetics - April 2017 - 18
Systems, Man & Cybernetics - April 2017 - 19
Systems, Man & Cybernetics - April 2017 - 20
Systems, Man & Cybernetics - April 2017 - 21
Systems, Man & Cybernetics - April 2017 - 22
Systems, Man & Cybernetics - April 2017 - 23
Systems, Man & Cybernetics - April 2017 - 24
Systems, Man & Cybernetics - April 2017 - 25
Systems, Man & Cybernetics - April 2017 - 26
Systems, Man & Cybernetics - April 2017 - 27
Systems, Man & Cybernetics - April 2017 - 28
Systems, Man & Cybernetics - April 2017 - 29
Systems, Man & Cybernetics - April 2017 - 30
Systems, Man & Cybernetics - April 2017 - 31
Systems, Man & Cybernetics - April 2017 - 32
Systems, Man & Cybernetics - April 2017 - 33
Systems, Man & Cybernetics - April 2017 - 34
Systems, Man & Cybernetics - April 2017 - 35
Systems, Man & Cybernetics - April 2017 - 36
Systems, Man & Cybernetics - April 2017 - 37
Systems, Man & Cybernetics - April 2017 - 38
Systems, Man & Cybernetics - April 2017 - 39
Systems, Man & Cybernetics - April 2017 - 40
Systems, Man & Cybernetics - April 2017 - 41
Systems, Man & Cybernetics - April 2017 - 42
Systems, Man & Cybernetics - April 2017 - 43
Systems, Man & Cybernetics - April 2017 - 44
Systems, Man & Cybernetics - April 2017 - 45
Systems, Man & Cybernetics - April 2017 - 46
Systems, Man & Cybernetics - April 2017 - 47
Systems, Man & Cybernetics - April 2017 - 48
Systems, Man & Cybernetics - April 2017 - 49
Systems, Man & Cybernetics - April 2017 - 50
Systems, Man & Cybernetics - April 2017 - 51
Systems, Man & Cybernetics - April 2017 - 52
Systems, Man & Cybernetics - April 2017 - 53
Systems, Man & Cybernetics - April 2017 - 54
Systems, Man & Cybernetics - April 2017 - 55
Systems, Man & Cybernetics - April 2017 - 56
Systems, Man & Cybernetics - April 2017 - Cover3
Systems, Man & Cybernetics - April 2017 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/smc_202310
https://www.nxtbook.com/nxtbooks/ieee/smc_202307
https://www.nxtbook.com/nxtbooks/ieee/smc_202304
https://www.nxtbook.com/nxtbooks/ieee/smc_202301
https://www.nxtbook.com/nxtbooks/ieee/smc_202210
https://www.nxtbook.com/nxtbooks/ieee/smc_202207
https://www.nxtbook.com/nxtbooks/ieee/smc_202204
https://www.nxtbook.com/nxtbooks/ieee/smc_202201
https://www.nxtbook.com/nxtbooks/ieee/smc_202110
https://www.nxtbook.com/nxtbooks/ieee/smc_202107
https://www.nxtbook.com/nxtbooks/ieee/smc_202104
https://www.nxtbook.com/nxtbooks/ieee/smc_202101
https://www.nxtbook.com/nxtbooks/ieee/smc_202010
https://www.nxtbook.com/nxtbooks/ieee/smc_202007
https://www.nxtbook.com/nxtbooks/ieee/smc_202004
https://www.nxtbook.com/nxtbooks/ieee/smc_202001
https://www.nxtbook.com/nxtbooks/ieee/smc_201910
https://www.nxtbook.com/nxtbooks/ieee/smc_201907
https://www.nxtbook.com/nxtbooks/ieee/smc_201904
https://www.nxtbook.com/nxtbooks/ieee/smc_201901
https://www.nxtbook.com/nxtbooks/ieee/smc_201810
https://www.nxtbook.com/nxtbooks/ieee/smc_201807
https://www.nxtbook.com/nxtbooks/ieee/smc_201804
https://www.nxtbook.com/nxtbooks/ieee/smc_201801
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1017
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0717
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0417
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0117
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1016
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0716
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0416
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0116
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_1015
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0715
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0415
https://www.nxtbook.com/nxtbooks/ieee/systems_man_cybernetics_0115
https://www.nxtbookmedia.com