IEEE Geoscience and Remote Sensing Magazine - December 2023 - 97

aforementioned efficiency metrics: number of parameters,
memory consumption, and latency, to achieve efficient inference
with a tradeoff of the model's quality [46]. Here, we
briefly summarize the commonly used model compression
techniques and refer the readers to [48], [49], and [50] for a
more detailed discussion on this subject.
COMPACT ARCHITECTURE
The fundamental approach to optimize a model for efficient
inference is to develop it with building blocks that allow for
greater efficiency in computation and memory footprints [46].
Convolutional layers are largely used building blocks [42],
[53], [59], [60]; thus, convolution operations contribute most
of the computation and memory footprints of a model [61].
Reducing the number of FLOPs in the convolution operations
of a model can significantly improve its efficiency [61]. Several
handcrafted convolutional building blocks have been proposed
to reduce the number of FLOPs in convolution operations,
and they are widely used to develop models for efficient
inference [48], [62]. They include depthwise separable convolution
[54], bottleneck convolution [51], inverted bottleneck
convolution [53], dilated convolution [63], grouped convolution
[52], [64], and asymmetric convolution [65]. The handcrafted
designs have achieved remarkable success; however,
handcrafted engineering is laborious and extremely dependent
on human expertise [66]. Hence, recent years have seen
a growing trend for automated machine learning (AutoML)
and neural architecture search (NAS) [67], [68] for automatically
searching for efficient cells (i.e., a directed acyclic graph
of convolutional layers) to develop efficient models [62], [69].
Figure 2(a)-(d) illustrates some examples of handcrafted convolutional
blocks, and Figure 2(e) and (f) shows some automated-learned
cells commonly used to develop efficient DNNs to
reduce the models' number of parameters and FLOPs.
PRUNING AND SPARSIFICATION
These techniques are used to remove " unimportant " weights
and neurons in a model to reduce its number of computation
arithmetic operations and make it smaller [70], [71], [72], [73]
[see Figure 2(g) and (h)]. This results in a reduction of memory
access, thus accelerating inference. Pruning algorithms have
been applied at different granularities of sparsity, from fine
grained (unstructured) to coarse grained (structured) [74]. For
fine-grained pruning, the individual weights or neurons of a
model are removed [75], [76], [77], whereas coarse-grained
pruning removes entire filters or channels [78], [79]. Unlike
fine-grained techniques, the coarse-grained pruning methods
are flexible to implement efficiently on hardware but can degrade
the model quality [80]; however, the pruned model can
be retrained to compensate the quality lost [81], [82]. AutoML
has also been employed to automatically prune the weights
and neurons of a model [83].
KNOWLEDGE DISTILLATION
The main idea is to compress a large model by transferring
the knowledge of the large model, often called the
DECEMBER 2023 IEEE GEOSCIENCE AND REMOTE SENSING MAGAZINE
teacher model, into a small model, also known as the student
model [57], [84] [see Figure 2(i)]. Basically, a small
model is trained using a large model as a supervisor to enable
the small model, which has less memory and energy
footprints, to mimic the quality of the large model for efficient
inference [85], [86], [87]. The learning algorithms
of knowledge distillation techniques in the literature [88],
[89] can be grouped as self-distillation [90], online distillation
[91], and offline distillation [92]. Recently, NAS has
been adopted in knowledge distillation [93], [94]. We refer
the readers to [95] for a detailed discussion on this subject.
QUANTIZATION AND BINARIZATION
While pruning reduces the number of weights and neurons in
a model, quantization methods aim to reduce the bit width
of the values of weights and neurons [48] [see Figure 2(j)-
(m)]. This normally renders a model with low-bit precision
operands [96], [97]. Quantization methods do not reduce
the number of computation arithmetic operations as in pruning
but simplify the operations, which therefore can shrink
the model size to reduce memory and energy footprints
and speed up inference but with some sacrifice of model
quality [98]. The commonly used methods in the literature
[99], [100], [101] include linear quantization methods [102],
K-means methods [103], and binary/ternary methods [104],
[105]. Besides these conventional methods, automated quantization
methods (i.e., quantization with AutoML) have been
proposed as well [106], [107]. It is not easy to implement
quantization in practice because it requires a good understanding
of bitwise algorithm and hardware architecture [7].
LOW-RANK APPROXIMATION
Here, the core idea is to reduce the computational complexity
of a model by approximating the redundant weight matrices
or tensors of a convolutional or fully connected layer
using a linear combination of fewer weights [108], [109].
Compressing a model in this fashion results in a significant
reduction in the model size, the model footprint, and the
inference latency [110] [see Figure 2(n) and (o)]. Various
methods adopted in the literature [7], [111], [112] include
singular value decomposition [113] and tensor decomposition
methods, such as Tucker decomposition [114], [115]
and canonical polyadic decomposition [116].
EFFICIENCY METRICS
Other than the quality of segmentation results, an efficient
model has three core characteristics: smaller, faster,
and greener, which are respectively gauged on storage, inference
speed, and energy [46], [117]. When designing or
choosing a DNN architecture for semantic segmentation
in real-time applications, one major consideration is the
quality-cost tradeoff [118]. Accuracy is used to measure
the quality of model results. The most commonly used
accuracy evaluation metric in semantic segmentation is
the IoU, known as the Jaccard index, and its variants mIoU
and frequency-weighted IoU (FwIoU) [25], [40] [see (1),
97
IEEE Geoscience and Remote Sensing Magazine - December 2023

Table of Contents for the Digital Edition of IEEE Geoscience and Remote Sensing Magazine - December 2023
