IEEE Circuits and Systems Magazine - Q3 2020 - 31

In DSC, computation is performed on "single-bit" sequences, so a significant
improvement in hardware efficiency is achieved with a high accuracy.
the Sobol sequence helps improving the accuracy. Fig. 26
compares the convergence of -m isalignment during
the training of the adaptive filter using pseudorandom
and quasirandom numbers. 16-bit stochastic integrators are used in this experiment. The misalignment is
given by
Misalignment =

E[(w - ~t )2 ]
, (15)
E[w 2 ]

where w denotes the target value and ~t is the trained
result. It can be seen that the convergence speed of using these two types of random numbers is similar and
their misalignments both converge to around -52 dB.
VI. Hardware Efficiency Evaluation
Due to the extremely low sequence length used for encoding a value in DSC, the power consumption of DSC
circuits is much lower than conventional arithmetic circuits. Moreover, since each bit in a DSS is used to encode a number, DSC is more energy-efficient than CSC.
The gradient descent and ODE solver circuits are conTable II.
Hardware evaluation of the gradient descent circuit
array training a 784-128-128-10 neural network.
Metrics

DSC

Fixed-Point

-10

Step size

Epochs

-10

Ratio
-

1.6 # 10

4.7 # 10

1:3

EPO (fJ)

1.2 # 10

1:9.2

TPA (image/s/ nm 2 )

5.7 # 10 1

1.5

38:1

Aver. test Accu.

97.04%

97.49%

Min. time (ns)

1.1 # 10

Table III.
Hardware performance comparison of the ODE
solvers for (8).
Metric

DSC
2

Step size

Fixed-Point

-8

Ratio
-

Iterations

3217

Min. time (ns)

2573.59

8557.20

1:3.3

201.21

466.00

1:2.3

EPO (fJ)
2

TPA (word/μs/ nm )
RMSE

4.75
4.7 # 10

0.58
-3

3.6 # 10

8.2:1
-3

THIRD QUARTER 2020

sidered as examples to show the hardware efficiency.
Implemented in VHDL, the circuits are synthesized by
Synopsys Design Compiler with a 28-nm technology to
obtain the power consumption, maximum frequency
and hardware measurements. The minimum runtime,
energy per operation (EPO), throughput per area (TPA)
and accuracy are reported in Table II for the gradient
descent circuits and in Table III for ODE solvers.
As can be seen, with a slightly degraded accuracy,
the DSC circuits outperform conventional circuits using
a fixed-point representation in speed, energy efficiency and hardware efficiency. For the gradient descent
circuit, since the two RNGs can be shared among the
118,282 stochastic integrators (for 118,282 weights), the
improvement in hardware efficiency is even more significant than the ODE solver, for which one RNG is shared
between two stochastic integrators. However, since the
sharing of RNGs does not significantly affect the critical path, the performance improvements are similar for
these two circuits.
VII. Conclusion
In this article, a new type of stochastic computing, dynamic stochastic computing (DSC), is introduced to encode a varying signal by using a dynamically variable
binary bit stream, referred to as a dynamic stochastic
sequence (DSS). A DSC circuit can efficiently implement
an iterative accumulation-based algorithm. It is useful
in implementing the Euler method, training a neural
network and an adaptive filter, and IIR filtering. In those
applications, integrations are implemented by stochastic integrators that accumulate the stochastic bits in a
DSS. Since the computation is performed on single stochastic bits instead of conventional binary values or
long stochastic sequences, a significant improvement
in hardware efficiency is achieved with a high accuracy.
Acknowledgment
This work was supported by the Natural Sciences and
Engineering Research Council of Canada (NSERC) under
Projects RES0025211 and RES0048688.
Siting Liu received the B.Eng. and M.
Eng. degrees in electrical engineering
and automation from the Harbin Institute of Technology, Harbin, Heilongjiang, China, in 2012 and 2014, respectively, and the Ph.D. degree from the
IEEE CIRCUITS AND SYSTEMS MAGAZINE

IEEE Circuits and Systems Magazine - Q3 2020

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q3 2020