IEEE Circuits and Systems Magazine - Q2 2023 - 14
CP-rank. RR Rd12
,, , are the Tucker-rank where d is
the Core-order. Figure 7 shows the block term decomposition
for a 3rd -order tensor.
G. Hierarchical Tucker Decomposition
Hierarchical Tucker decomposition has multiple hierarchical
levels based on the order of the tensor. From
top to bottom in a binary tree, a hierarchical tucker
decomposed tensor can be recursively decomposed
into intermediate tensors, referred as frames, as shown
in Fig. 8. Each frame is a unique node where each node
is associated with a dimension set. Given a d -dimensional
target tensor X∈
R ×××
II Id12
a root node associated with Dd={}
rIµν×× ×I
12
∈R⊊
,
the left and right child nodes of the s -associated node
where µs
= mi ()n
s
((G∈
R ××
rr rss s12
and νs = x. A non-leaf frame
ma ()
s
Us can be recursively decomposed to a left child frame
() Us1
, a right child frame Us2() and a transfer tensor
)) as follows [34]:
UG UUss ss=× ×
1
2
12,
1
2
where ×1
2 denotes the tensor contraction that can be executed
between two tensors with at least one matched dimension.
For example, given two tensors A∈
R ××
B∈R ××
, a binary tree with
,, , can be built
where XU= D is the root frame. For each non-leaf frame
Us
s ss sD is associated with the node corresponding
to Us and ss s12, ⊊ are associated with
lm m12
nn l12
and
where the third dimension of A matches
×× ×
AB ,, ,,,, ,AB×=∑ =
1
3
ii jj αα jjα 12 . The origil
1
212
1
ii
12
the first dimension of B, a tensor of size nn mm1 212
can be computed by using the tensor contraction operation
as ()
nal IId
1×× -order tensor XU= D can be recursively
decomposed into a combination of the 2nd -order leaf
frames and the 3rd -order transfer
tensors by performing the hierarchical
tucker decomposition from
top to bottom of the binary tree.
The parameter rs is defined as hierarchical
rank.
III. Tensorizing Convolutional
Neural Networks
Figure 7. Block Term decomposition for a 3rd-order tensor. The tensor can be approximated
by N Tucker decompositions where N is the CP-rank; R1, R2, R3 are the
Tucker-rank; d is the Core-order [33].
CNNs have two main parts: convolutional
layers and fully connected
layers. In general, convolutional
layers in CNNs map a third-order
input tensor X of size SW H××
into a third-order output tensor
Y of size TW H××′′ with a 4th
order kernel tensor K of size
TS DD×× × , where DD×
represents
the filter size, S and T
represent the number of input and
output channels, respectively. The
typical convolutional filter sizes
are small, e.g.,
33 77 , com××,
pared
to the numbers of input S()
and output T() channels. Each
convolutional layer may have hundreds
or thousands of filters which
are suited for tensor decomposition
methods. As shown in Fig. 9,
the output tensor Y can be computed
as:
Figure 8. Hierarchical Tucker decomposition for a 4th-order tensor. It is a binary tree
with root D = {1, 2, 3, 4} where the dashed boxes represent the nodes. Node {1} is a
leaf node whose parent and sibling are node {1, 2} and node {2}, respectively [34].
14
IEEE CIRCUITS AND SYSTEMS MAGAZINE
S
tw h
,,
D
j
i
D
YK Xts ji sw hji
′′ =
∑∑∑
s 11 1== =
SECOND QUARTER 2023
,, ,, ,
,
IEEE Circuits and Systems Magazine - Q2 2023
Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q2 2023
Contents
IEEE Circuits and Systems Magazine - Q2 2023 - Cover1
IEEE Circuits and Systems Magazine - Q2 2023 - Cover2
IEEE Circuits and Systems Magazine - Q2 2023 - Contents
IEEE Circuits and Systems Magazine - Q2 2023 - 2
IEEE Circuits and Systems Magazine - Q2 2023 - 3
IEEE Circuits and Systems Magazine - Q2 2023 - 4
IEEE Circuits and Systems Magazine - Q2 2023 - 5
IEEE Circuits and Systems Magazine - Q2 2023 - 6
IEEE Circuits and Systems Magazine - Q2 2023 - 7
IEEE Circuits and Systems Magazine - Q2 2023 - 8
IEEE Circuits and Systems Magazine - Q2 2023 - 9
IEEE Circuits and Systems Magazine - Q2 2023 - 10
IEEE Circuits and Systems Magazine - Q2 2023 - 11
IEEE Circuits and Systems Magazine - Q2 2023 - 12
IEEE Circuits and Systems Magazine - Q2 2023 - 13
IEEE Circuits and Systems Magazine - Q2 2023 - 14
IEEE Circuits and Systems Magazine - Q2 2023 - 15
IEEE Circuits and Systems Magazine - Q2 2023 - 16
IEEE Circuits and Systems Magazine - Q2 2023 - 17
IEEE Circuits and Systems Magazine - Q2 2023 - 18
IEEE Circuits and Systems Magazine - Q2 2023 - 19
IEEE Circuits and Systems Magazine - Q2 2023 - 20
IEEE Circuits and Systems Magazine - Q2 2023 - 21
IEEE Circuits and Systems Magazine - Q2 2023 - 22
IEEE Circuits and Systems Magazine - Q2 2023 - 23
IEEE Circuits and Systems Magazine - Q2 2023 - 24
IEEE Circuits and Systems Magazine - Q2 2023 - 25
IEEE Circuits and Systems Magazine - Q2 2023 - 26
IEEE Circuits and Systems Magazine - Q2 2023 - 27
IEEE Circuits and Systems Magazine - Q2 2023 - 28
IEEE Circuits and Systems Magazine - Q2 2023 - 29
IEEE Circuits and Systems Magazine - Q2 2023 - 30
IEEE Circuits and Systems Magazine - Q2 2023 - 31
IEEE Circuits and Systems Magazine - Q2 2023 - 32
IEEE Circuits and Systems Magazine - Q2 2023 - 33
IEEE Circuits and Systems Magazine - Q2 2023 - 34
IEEE Circuits and Systems Magazine - Q2 2023 - 35
IEEE Circuits and Systems Magazine - Q2 2023 - 36
IEEE Circuits and Systems Magazine - Q2 2023 - 37
IEEE Circuits and Systems Magazine - Q2 2023 - 38
IEEE Circuits and Systems Magazine - Q2 2023 - 39
IEEE Circuits and Systems Magazine - Q2 2023 - 40
IEEE Circuits and Systems Magazine - Q2 2023 - 41
IEEE Circuits and Systems Magazine - Q2 2023 - 42
IEEE Circuits and Systems Magazine - Q2 2023 - 43
IEEE Circuits and Systems Magazine - Q2 2023 - 44
IEEE Circuits and Systems Magazine - Q2 2023 - 45
IEEE Circuits and Systems Magazine - Q2 2023 - 46
IEEE Circuits and Systems Magazine - Q2 2023 - 47
IEEE Circuits and Systems Magazine - Q2 2023 - 48
IEEE Circuits and Systems Magazine - Q2 2023 - 49
IEEE Circuits and Systems Magazine - Q2 2023 - 50
IEEE Circuits and Systems Magazine - Q2 2023 - 51
IEEE Circuits and Systems Magazine - Q2 2023 - 52
IEEE Circuits and Systems Magazine - Q2 2023 - 53
IEEE Circuits and Systems Magazine - Q2 2023 - 54
IEEE Circuits and Systems Magazine - Q2 2023 - 55
IEEE Circuits and Systems Magazine - Q2 2023 - 56
IEEE Circuits and Systems Magazine - Q2 2023 - 57
IEEE Circuits and Systems Magazine - Q2 2023 - 58
IEEE Circuits and Systems Magazine - Q2 2023 - 59
IEEE Circuits and Systems Magazine - Q2 2023 - 60
IEEE Circuits and Systems Magazine - Q2 2023 - 61
IEEE Circuits and Systems Magazine - Q2 2023 - 62
IEEE Circuits and Systems Magazine - Q2 2023 - 63
IEEE Circuits and Systems Magazine - Q2 2023 - 64
IEEE Circuits and Systems Magazine - Q2 2023 - 65
IEEE Circuits and Systems Magazine - Q2 2023 - 66
IEEE Circuits and Systems Magazine - Q2 2023 - 67
IEEE Circuits and Systems Magazine - Q2 2023 - 68
IEEE Circuits and Systems Magazine - Q2 2023 - 69
IEEE Circuits and Systems Magazine - Q2 2023 - 70
IEEE Circuits and Systems Magazine - Q2 2023 - 71
IEEE Circuits and Systems Magazine - Q2 2023 - 72
IEEE Circuits and Systems Magazine - Q2 2023 - Cover3
IEEE Circuits and Systems Magazine - Q2 2023 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2023Q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2023Q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2023Q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021Q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q1
https://www.nxtbookmedia.com