IEEE Circuits and Systems Magazine - Q2 2023 - 22

of size 33× . Each convolutional layer excluding
the first one was replaced by the TT-conv layer as
mentioned before. The authors also compared the
proposed Tensor Train decomposition method for
the convolutional layer with the naive approach
which directly applies the Tensor Train decomposition
to the 4th -order convolutional kernel. As
illustrated in Table 2, the proposed approach can
achieve similar accuracies as the naive baseline
on the 2× compression level. The second network
was adapted from the first one by replacing the
average pooling with two fully connected layers of
size 8192 1536×
and 1536 512×
. Initially, the fullyconnected
part was the memory bottleneck. As
shown in Table 3, a 21 01. × network compression
with 07
.% accuracy drop can be achieved by only
replacing the fully connected layers with TT-FC
layers as described before. Then the bottleneck
moves to the convolutional part. At this point,
by additionally replacing the convolutional layTable
2.
Compressing convolution-dominated CNN on the
CIFAR-10 dataset [21]. Different rows with the same
model name represent different choices of the
TT-ranks. CR stands for compression rate.
Model
conv (original)
TT-conv
TT-conv
TT-conv
TT-conv
TT-conv (naive)
TT-conv (naive)
Top-1 accuracy
90.7%
89.9%
89.2%
89.3%
88.7%
88.3%
87.6%
CR
-
2.02×
2.53×
3.23×
4.02×
2.02×
2.90×
Table 3.
Compressing FC-dominated CNN on the CIFAR-10
dataset [21]. Different rows with the same model
name represent different choices of the TT-ranks.
CR stands for compression rate.
Model
conv-FC
(original)
conv-TT-FC
conv-TT-FC
conv-TT-FC
TT-conv-TT-FC
TT-conv-TT-FC
TT-conv-TT-FC
22
Top-1 accuracy
90.5%
90.3%
89.8%
89.8%
90.1%
89.7%
89.4%
IEEE CIRCUITS AND SYSTEMS MAGAZINE
CR
-
10.72×
19.38×
21.01×
9.69×
41.65×
82.87×
ers with the TT-conv layers, an 82 87. × network
compression with 11
.% accuracy drop can be
achieved as shown in Table 3. More details can be
found in [21].
2) CP Decomposition: In [17], CP decomposition
was implemented in two steps: applying CP decomposition
to the convolutional layer using the
NLS algorithm and then finetuning the entire
network using backpropagation. Two network
architectures were tested: small character-classification
CNN from [68] and AlexNet [1]. The
character-classification CNN, called CharNet,
has four convolutional layers. This CNN was
used to classify images of size 24 24×
into one
of 36 classes (10 digits +26 characters). Only the
second and the third convolutional layers were
compressed since they cost more than 90% of
processing time. The second layer has 48 input
channels and 128 output channels with filters
of size 99× . The third layer has 48 input channels
and 128 output channels with filters of size
88× . First, the second layer was compressed
using CP decomposition with rank 64. Then all
layers but the new ones were fine-tuned to reduce
the accuracy drop. Finally, the third layer
was approximated using CP decomposition with
rank 64. Since the last approximation does not
lead to a large accuracy drop, there is no need to
fine-tune the network after that. The compressed
network is 8.5 times faster than the original one
while the classification accuracy only drops by
1% to 90 2.%. AlexNet is one of the common object
recognition networks and it has eight layers
consisting of five convolution layers and three
fully connected layers. The second convolutional
layer of AlexNet was compressed in [17] by using
CP decomposition. The running time of the
second layer can be accelerated by 36. × using
a rank of 200 at the expense of 05
.% accuracy
degradation or by 45. × with a rank of 140 at the
expense of ≈1% accuracy degradation. Another
fact mentioned in [17] is that greedy CP decompositions
like ALS work worse than NLS for CNN
model compression.
Tensor Power Method (TPM) [69] was used to apply
CP decomposition to the convolutional kernels
in [15]. Compared to ALS, TPM can achieve the same
variance with less rank since the rank-1 tensors found
in the early steps of TPM represents most of the variances
in the original tensor. TPM compressed the
convolutional kernels by adding rank-1 tensors until
a predefined number of rank-1 tensors is found. First,
TPM finds a rank-1 tensor K1 by minimizing KK−
1 2 .
SECOND QUARTER 2023
IEEE Circuits and Systems Magazine - Q2 2023

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q2 2023
