IEEE Circuits and Systems Magazine - Q3 2023 - 21

manner. For each time, it only runs the model on a
small spatial region (>10× smaller than the whole area),
which effectively cuts down the peak memory usage.
After this stage is finished, the rest of the network with
a small peak memory is executed in a normal layer-bylayer
manner (upper notations in Figure 11).
An example of two convolutional layers (with
stride 1 and 2) is shown in Figure 12. For conventional
Figure 10. TinyEngine reduces peak memory by performing in-place depth-wise convolution. Conventional depth-wise convolution
requires 2N memory footprint for activations (left). In-place depth-wise convolution reduces the memory of depth-wise
convolutions to N+1 (right). Specifically, the output activation of the first channel is stored in a temporary buffer. Then, for each
following channel, the output activation overwrites the input activation of its previous channel. Finally, the output activation of
the first channel stored in the buffer is written back to the input activation of the last channel. (a) Depth-wise convolution. (b)
In-place depth-wise convolution.
Figure 11. MobileNetV2 [4] has a very imbalanced memory usage distribution. The peak memory is determined by the first
five blocks with high peak memory, while the later blocks all share a small memory usage. By using per-patch inference (4 × 4
patches), we are able to significantly reduce the memory usage of the first five blocks, and reduce the overall peak memory by
8×, fitting MCUs with a 256kB memory budget. Notice that the model architecture and accuracy are not changed for the two
settings. The memory usage is measured in int8.
Figure 12. Per-patch inference can significantly reduce the peak memory required to execute a sequence of convolutional
layers. We study two convolutional layers (stride 1 and 2). (a) Under per-layer computation, the first convolution has a large
input/output activation size, dominating the peak memory requirement. (b) With per-patch computation, we allocate the buffer
to host the final output activation, and compute the results patch-by-patch. We only need to store the activation from one patch
but not the entire feature map, reducing the peak memory (the first input is the image, which can be partially decoded from a
compressed format like JPEG).
THIRD QUARTER 2023
IEEE CIRCUITS AND SYSTEMS MAGAZINE
21

IEEE Circuits and Systems Magazine - Q3 2023

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q3 2023

Contents
IEEE Circuits and Systems Magazine - Q3 2023 - Cover1
IEEE Circuits and Systems Magazine - Q3 2023 - Cover2
IEEE Circuits and Systems Magazine - Q3 2023 - Contents
IEEE Circuits and Systems Magazine - Q3 2023 - 2
IEEE Circuits and Systems Magazine - Q3 2023 - 3
IEEE Circuits and Systems Magazine - Q3 2023 - 4
IEEE Circuits and Systems Magazine - Q3 2023 - 5
IEEE Circuits and Systems Magazine - Q3 2023 - 6
IEEE Circuits and Systems Magazine - Q3 2023 - 7
IEEE Circuits and Systems Magazine - Q3 2023 - 8
IEEE Circuits and Systems Magazine - Q3 2023 - 9
IEEE Circuits and Systems Magazine - Q3 2023 - 10
IEEE Circuits and Systems Magazine - Q3 2023 - 11
IEEE Circuits and Systems Magazine - Q3 2023 - 12
IEEE Circuits and Systems Magazine - Q3 2023 - 13
IEEE Circuits and Systems Magazine - Q3 2023 - 14
IEEE Circuits and Systems Magazine - Q3 2023 - 15
IEEE Circuits and Systems Magazine - Q3 2023 - 16
IEEE Circuits and Systems Magazine - Q3 2023 - 17
IEEE Circuits and Systems Magazine - Q3 2023 - 18
IEEE Circuits and Systems Magazine - Q3 2023 - 19
IEEE Circuits and Systems Magazine - Q3 2023 - 20
IEEE Circuits and Systems Magazine - Q3 2023 - 21
IEEE Circuits and Systems Magazine - Q3 2023 - 22
IEEE Circuits and Systems Magazine - Q3 2023 - 23
IEEE Circuits and Systems Magazine - Q3 2023 - 24
IEEE Circuits and Systems Magazine - Q3 2023 - 25
IEEE Circuits and Systems Magazine - Q3 2023 - 26
IEEE Circuits and Systems Magazine - Q3 2023 - 27
IEEE Circuits and Systems Magazine - Q3 2023 - 28
IEEE Circuits and Systems Magazine - Q3 2023 - 29
IEEE Circuits and Systems Magazine - Q3 2023 - 30
IEEE Circuits and Systems Magazine - Q3 2023 - 31
IEEE Circuits and Systems Magazine - Q3 2023 - 32
IEEE Circuits and Systems Magazine - Q3 2023 - 33
IEEE Circuits and Systems Magazine - Q3 2023 - 34
IEEE Circuits and Systems Magazine - Q3 2023 - 35
IEEE Circuits and Systems Magazine - Q3 2023 - 36
IEEE Circuits and Systems Magazine - Q3 2023 - 37
IEEE Circuits and Systems Magazine - Q3 2023 - 38
IEEE Circuits and Systems Magazine - Q3 2023 - 39
IEEE Circuits and Systems Magazine - Q3 2023 - 40
IEEE Circuits and Systems Magazine - Q3 2023 - 41
IEEE Circuits and Systems Magazine - Q3 2023 - 42
IEEE Circuits and Systems Magazine - Q3 2023 - 43
IEEE Circuits and Systems Magazine - Q3 2023 - 44
IEEE Circuits and Systems Magazine - Q3 2023 - 45
IEEE Circuits and Systems Magazine - Q3 2023 - 46
IEEE Circuits and Systems Magazine - Q3 2023 - 47
IEEE Circuits and Systems Magazine - Q3 2023 - 48
IEEE Circuits and Systems Magazine - Q3 2023 - 49
IEEE Circuits and Systems Magazine - Q3 2023 - 50
IEEE Circuits and Systems Magazine - Q3 2023 - 51
IEEE Circuits and Systems Magazine - Q3 2023 - 52
IEEE Circuits and Systems Magazine - Q3 2023 - 53
IEEE Circuits and Systems Magazine - Q3 2023 - 54
IEEE Circuits and Systems Magazine - Q3 2023 - 55
IEEE Circuits and Systems Magazine - Q3 2023 - 56
IEEE Circuits and Systems Magazine - Q3 2023 - 57
IEEE Circuits and Systems Magazine - Q3 2023 - 58
IEEE Circuits and Systems Magazine - Q3 2023 - 59
IEEE Circuits and Systems Magazine - Q3 2023 - 60
IEEE Circuits and Systems Magazine - Q3 2023 - 61
IEEE Circuits and Systems Magazine - Q3 2023 - 62
IEEE Circuits and Systems Magazine - Q3 2023 - 63
IEEE Circuits and Systems Magazine - Q3 2023 - 64
IEEE Circuits and Systems Magazine - Q3 2023 - 65
IEEE Circuits and Systems Magazine - Q3 2023 - 66
IEEE Circuits and Systems Magazine - Q3 2023 - 67
IEEE Circuits and Systems Magazine - Q3 2023 - 68
IEEE Circuits and Systems Magazine - Q3 2023 - 69
IEEE Circuits and Systems Magazine - Q3 2023 - 70
IEEE Circuits and Systems Magazine - Q3 2023 - 71
IEEE Circuits and Systems Magazine - Q3 2023 - 72
IEEE Circuits and Systems Magazine - Q3 2023 - 73
IEEE Circuits and Systems Magazine - Q3 2023 - 74
IEEE Circuits and Systems Magazine - Q3 2023 - 75
IEEE Circuits and Systems Magazine - Q3 2023 - 76
IEEE Circuits and Systems Magazine - Q3 2023 - 77
IEEE Circuits and Systems Magazine - Q3 2023 - 78
IEEE Circuits and Systems Magazine - Q3 2023 - 79
IEEE Circuits and Systems Magazine - Q3 2023 - 80
IEEE Circuits and Systems Magazine - Q3 2023 - 81
IEEE Circuits and Systems Magazine - Q3 2023 - 82
IEEE Circuits and Systems Magazine - Q3 2023 - 83
IEEE Circuits and Systems Magazine - Q3 2023 - 84
IEEE Circuits and Systems Magazine - Q3 2023 - 85
IEEE Circuits and Systems Magazine - Q3 2023 - 86
IEEE Circuits and Systems Magazine - Q3 2023 - 87
IEEE Circuits and Systems Magazine - Q3 2023 - 88
IEEE Circuits and Systems Magazine - Q3 2023 - Cover3
IEEE Circuits and Systems Magazine - Q3 2023 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2023Q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2023Q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2023Q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021Q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q1
https://www.nxtbookmedia.com