IEEE Circuits and Systems Magazine - Q4 2022 - 41

resolution networks with nearly no accuracy loss relative
to floating-point networks [12], [59], [70].
Importantly,
the broad applicability of low-precision networks
increases the likelihood that QAT methods can be integrated
into state-of-the-art training workflows.
This section evaluates a 4-bit QAT network with the
ResNet50v1.5 topology, submitted by Nvidia to the MLPerf
Inference Benchmark [1]. The network uses 4-bit
weights and activations in all layers except the first and
last, which use 8-bit weights. Each ReLU output is multiplied
by 16-bit scaling factors before quantizing to 4 bits.
The digital software accuracy of this network is shown
in Table 2. When simulating the analog accuracy of this
network, these scaling steps are processed digitally
between in situ MVMs.
Fig. 19 compares the error sensitivity of the 4-bit QAT
model with the floating-point ResNet50-v1.5 model,
whose weights are quantized to 8 bits after training.
For a fair comparison, an 8-bit ADC is included for both
cases; in the 4-bit model, this higher-resolution ADC
helps minimize errors prior to the 4-bit quantization
step, which is performed digitally. Fig. 19(a) shows
that the 4-bit model is substantially more resilient to
state-independent errors than the 8-bit network. This
results entirely from activation quantization, and not
weight quantization. The large separation between the
16 activation levels effectively cuts off the propagation
of accumulated cell errors from one layer to the next.
The same cell error results in a smaller dot product
error on average.

IEEE Circuits and Systems Magazine - Q4 2022

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q4 2022

Contents
IEEE Circuits and Systems Magazine - Q4 2022 - Cover1
IEEE Circuits and Systems Magazine - Q4 2022 - Cover2
IEEE Circuits and Systems Magazine - Q4 2022 - Contents
IEEE Circuits and Systems Magazine - Q4 2022 - 2
IEEE Circuits and Systems Magazine - Q4 2022 - 3
IEEE Circuits and Systems Magazine - Q4 2022 - 4
IEEE Circuits and Systems Magazine - Q4 2022 - 5
IEEE Circuits and Systems Magazine - Q4 2022 - 6
IEEE Circuits and Systems Magazine - Q4 2022 - 7
IEEE Circuits and Systems Magazine - Q4 2022 - 8
IEEE Circuits and Systems Magazine - Q4 2022 - 9
IEEE Circuits and Systems Magazine - Q4 2022 - 10
IEEE Circuits and Systems Magazine - Q4 2022 - 11
IEEE Circuits and Systems Magazine - Q4 2022 - 12
IEEE Circuits and Systems Magazine - Q4 2022 - 13
IEEE Circuits and Systems Magazine - Q4 2022 - 14
IEEE Circuits and Systems Magazine - Q4 2022 - 15
IEEE Circuits and Systems Magazine - Q4 2022 - 16
IEEE Circuits and Systems Magazine - Q4 2022 - 17
IEEE Circuits and Systems Magazine - Q4 2022 - 18
IEEE Circuits and Systems Magazine - Q4 2022 - 19
IEEE Circuits and Systems Magazine - Q4 2022 - 20
IEEE Circuits and Systems Magazine - Q4 2022 - 21
IEEE Circuits and Systems Magazine - Q4 2022 - 22
IEEE Circuits and Systems Magazine - Q4 2022 - 23
IEEE Circuits and Systems Magazine - Q4 2022 - 24
IEEE Circuits and Systems Magazine - Q4 2022 - 25
IEEE Circuits and Systems Magazine - Q4 2022 - 26
IEEE Circuits and Systems Magazine - Q4 2022 - 27
IEEE Circuits and Systems Magazine - Q4 2022 - 28
IEEE Circuits and Systems Magazine - Q4 2022 - 29
IEEE Circuits and Systems Magazine - Q4 2022 - 30
IEEE Circuits and Systems Magazine - Q4 2022 - 31
IEEE Circuits and Systems Magazine - Q4 2022 - 32
IEEE Circuits and Systems Magazine - Q4 2022 - 33
IEEE Circuits and Systems Magazine - Q4 2022 - 34
IEEE Circuits and Systems Magazine - Q4 2022 - 35
IEEE Circuits and Systems Magazine - Q4 2022 - 36
IEEE Circuits and Systems Magazine - Q4 2022 - 37
IEEE Circuits and Systems Magazine - Q4 2022 - 38
IEEE Circuits and Systems Magazine - Q4 2022 - 39
IEEE Circuits and Systems Magazine - Q4 2022 - 40
IEEE Circuits and Systems Magazine - Q4 2022 - 41
IEEE Circuits and Systems Magazine - Q4 2022 - 42
IEEE Circuits and Systems Magazine - Q4 2022 - 43
IEEE Circuits and Systems Magazine - Q4 2022 - 44
IEEE Circuits and Systems Magazine - Q4 2022 - 45
IEEE Circuits and Systems Magazine - Q4 2022 - 46
IEEE Circuits and Systems Magazine - Q4 2022 - 47
IEEE Circuits and Systems Magazine - Q4 2022 - 48
IEEE Circuits and Systems Magazine - Q4 2022 - 49
IEEE Circuits and Systems Magazine - Q4 2022 - 50
IEEE Circuits and Systems Magazine - Q4 2022 - 51
IEEE Circuits and Systems Magazine - Q4 2022 - 52
IEEE Circuits and Systems Magazine - Q4 2022 - 53
IEEE Circuits and Systems Magazine - Q4 2022 - 54
IEEE Circuits and Systems Magazine - Q4 2022 - 55
IEEE Circuits and Systems Magazine - Q4 2022 - 56
IEEE Circuits and Systems Magazine - Q4 2022 - 57
IEEE Circuits and Systems Magazine - Q4 2022 - 58
IEEE Circuits and Systems Magazine - Q4 2022 - 59
IEEE Circuits and Systems Magazine - Q4 2022 - 60
IEEE Circuits and Systems Magazine - Q4 2022 - 61
IEEE Circuits and Systems Magazine - Q4 2022 - 62
IEEE Circuits and Systems Magazine - Q4 2022 - 63
IEEE Circuits and Systems Magazine - Q4 2022 - 64
IEEE Circuits and Systems Magazine - Q4 2022 - 65
IEEE Circuits and Systems Magazine - Q4 2022 - 66
IEEE Circuits and Systems Magazine - Q4 2022 - 67
IEEE Circuits and Systems Magazine - Q4 2022 - 68
IEEE Circuits and Systems Magazine - Q4 2022 - 69
IEEE Circuits and Systems Magazine - Q4 2022 - 70
IEEE Circuits and Systems Magazine - Q4 2022 - 71
IEEE Circuits and Systems Magazine - Q4 2022 - 72
IEEE Circuits and Systems Magazine - Q4 2022 - 73
IEEE Circuits and Systems Magazine - Q4 2022 - 74
IEEE Circuits and Systems Magazine - Q4 2022 - 75
IEEE Circuits and Systems Magazine - Q4 2022 - 76
IEEE Circuits and Systems Magazine - Q4 2022 - 77
IEEE Circuits and Systems Magazine - Q4 2022 - 78
IEEE Circuits and Systems Magazine - Q4 2022 - 79
IEEE Circuits and Systems Magazine - Q4 2022 - 80
IEEE Circuits and Systems Magazine - Q4 2022 - Cover3
IEEE Circuits and Systems Magazine - Q4 2022 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2023Q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2023Q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2023Q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2022Q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021Q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2021q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2020q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2019q1
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q4
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q3
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q2
https://www.nxtbook.com/nxtbooks/ieee/circuitsandsystems_2018q1
https://www.nxtbookmedia.com