IEEE Circuits and Systems Magazine - Q3 2023 - 18

model with the highest accuracy while satisfying the resource
constraints.
Finding S* is non-trivial. One way is to perform neural
architecture search on each of the search spaces and
compare the final results. But the computation would
be astronomical. Instead, TinyNAS evaluates the quality
of the search space by randomly sampling m networks
from the search space and comparing the distribution of
satisfying networks. Instead of collecting the cumulative
distribution function (CDF) of each satisfying network's
accuracy [88], which is computationally heavy due to
tremendous training, it only collects the CDF of FLOPs
(see Figure 5(b)). The intuition is that, within the same
model family, the accuracy is usually positively related
to the computation [61], [128]. A model with larger computation
has a larger capacity, which is more likely to
achieve higher accuracy.
Take the study of the best search space for ImageNet-100
(a 100-class classification task taken from the
original ImageNet) on STM32F746 as an example. We
show the FLOPs distribution CDF of the top-10 search
space configurations in Figure 5(b). Only the models
that satisfy the memory requirement at the best scheduling
from TinyEngine are kept. For example, according
to the experimental results on ImageNet-100, using the
solid red space (average FLOPs 52.0 M) achieves 2.3%
better accuracy compared to using the solid green space
(average FLOPs 46.9 M), showing the effectiveness of automated
search space optimization.
2) Resource-Constrained Model
Specialization With Once-For-All NAS
To specialize network architecture for various microcontrollers,
we need to keep a low neural architecture search
cost. Given an optimized search space, TinyNAS further
performs one-shot neural architecture search [130], [131]
to efficiently find a good model. Specifically, it follows
once-for-all (OFA) NAS [129] to perform network specialization
(Figure 6). We train one super network that contains
all the possible sub-networks through weight sharing
and use it to estimate the performance of each sub-network.
The search space is based on the widely-used
mobile search space [85], [86], [87], [129] and supports
variable kernel sizes for depth-wise convolution (3/5/7),
variable
expansion raFigure
4. MCUNet jointly designs the neural architecture and the inference scheduling to fit the
tight memory resource on microcontrollers. TinyEngine makes full use of the limited resources
on MCU, allowing a larger design space for architecture search. With a larger degree of design
freedom, TinyNAS is more likely to find a high accuracy model compared to using existing frameworks.
(a) Search NN model on an existing library. (b) Tune deep learning library given a NN
model. (c) MCUNet: system-algorithm co-design.
tios for inverted bottleneck
(3/4/6) and variable
stage depths (2/3/4). The
number of possible subnetworks
that TinyNAS
can cover in the search
space is large: 2 × 1019.
For each batch of data, it
randomly samples four
sub-networks, calculates
the loss, backpropagates
the gradients for each
sub-network, and updates
the corresponding
Figure 5. (a) TinyNAS is a two-stage neural architecture search method. It first specifies a sub-space according to the constraints,
and then performs model specialization. (b) TinyNAS selects the best search space by analyzing the FLOPs CDF of
different search spaces. Each curve represents a design space. Our insight is that the design space that is more likely to produce
high FLOPs models under the memory constraint gives higher model capacity, thus more likely to achieve high accuracy.
18
IEEE CIRCUITS AND SYSTEMS MAGAZINE
THIRD QUARTER 2023

IEEE Circuits and Systems Magazine - Q3 2023

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q3 2023