IEEE Circuits and Systems Magazine - Q3 2020 - 30

Applications of DSC include the implementation of digital signal
processing (DSP) algorithms, ordinary differential equation (ODE)
solvers and the training of machine learning models.
reduces the variance, thus reducing the error. This is
shown as follows.
The error introduced by DSC at each step of computation is related to the variance of the difference of the
two accumulated stochastic bits, A i - B i . When independent RNGs are used to generate A i and B i , the variance at a single step is given by
Var [A i - B i] = E[(A i - B i - E (A i - B i)) 2]
= E[(A i - B i) 2] - 2E[A i - B i]
(PA, i - PB, i) + (PA, i - PB, i) 2,

	

(12)

where PA, i and PB, i are the probabilities of A i and B i being '1', respectively, i.e., E[A i] = PA, i and E[B i] = PB, i . Since
(A i - B i)2 is 0 when A i and B i are equal or 1 when they are
different, E[(A i - B i)2] equals PA, i (1 - PB, i) + PB, i (1 - PA, i),
which is the probability that A i and B i are different
when A i and B i are independently generated1. (12) is
then rewritten as
Varind [A i - B i] = PA, i (1 - PB, i) + PB, i (1 - PA, i) - (PA, i - PB, i) 2
(13)
= PA, i (1 - PA, i) + PB, i (1 - PB, i) .
On the other hand, if the same RNG is used to generate A i and B i, the variance of A i - B i can be derived
from (12) as well. However, since only a shared random
1 

2

2

2

E [(A i - B i) ] = Prob [(A i - B i) = 1] # 1 + Prob [(A i - B i) = 0] # 0

0

Pseudorandom
Quasirandom

Misalignment (dB)

-10
-20
-30
-40
-50
-60

0

0.5
1
1.5
2
2.5
3
Number of Steps (×64 Input Samples)
×104

Figure 26. Comparison of the convergence of misalignment
using pseudorandom and quasirandom numbers.
30 	

number is used to generate A i and B i, E[(A i - B i)2 ]
equals ; PA, i - PB, i ;, which gives the -probability that A i
and B i are different. Thus, (12) can be rewritten as
	

Varshare [A i - B i] = ; PA, i - PB, i ; - (PA, i - PB, i)2
	
= ; PA, i - PB, i ; ( 1 - ; PA, i - PB, i ; ).

(14)

Since Varshare [A i - B i] - Varind [A i - B i] = 2PB, i PA, i - 2 min
(PA, i P B, i) # 0, we obtain Varshare [A i - B i] # Varind [A i B i] for any i = 0, 1, 2, f [54]. Therefore, sharing the use
of RNGs to generate input stochastic bit streams improves the accuracy.
For the circuit in Fig. 22, though the inputs of the stochastic integrator are the outputs of the stochastic multipliers instead of being directly generated by a DSNG, the
RNG sharing scheme still works in the generated DSS's
encoding d + and d -, i.e., it produces a smaller variance in
the computed result [56]. However, an uncorrelated RNG
must be used to generate the DSS encoding the recorded
output of the neuron, o, because independency is still required in general for accurate stochastic multiplication.
The accuracy of the accumulated result is also related
to the bit-width of the counter in the stochastic integrator. The bit width does not only determine the resolution
of the result but also the step size, especially in the ODE
solver, adaptive filter and the gradient descent circuit.
A larger bit width indicates a smaller resolution and a
smaller step size for fine tuning at the cost of extra computation time. It is shown in [56] that the multi-step variance bound exponentially decreases with the bit width
of the counter. However, for the IIR filter, the change of
bit-width affects the characteristic of the filter.
For the ODE solver circuit, it is also found that the
use of quasirandom numbers improves the accuracy
compared to the use of pseudorandom numbers [54].
However, when the DSS is used to encode signals from
the input or images such as in the adaptive filter and
the gradient descent algorithm, quasirandom and
pseudorandom numbers produce similar accuracy.
The reason may lie in the fact that the values of the
encoded signals in these two applications are independently selected from the training dataset. However,
for the ODE solver, the signals encoded are samples
from continuous functions, and adjacent samples have
similar values. As a result, within a short segment of
the DSS in this case, it approximately encodes the
same value. It then works similarly as CSC, for which

IEEE CIRCUITS AND SYSTEMS MAGAZINE 		

THIRD QUARTER 2020
IEEE Circuits and Systems Magazine - Q3 2020

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q3 2020
