IEEE Circuits and Systems Magazine - Q1 2021 - 19

Stochastic Computing and Hyper-Dimensional Computing randomness improves
the robustness of circuits subject to faults or component variability.

pi = c

p^ X h i
m
1 - p^ X h

(25)

and by applying (21) to normalize (25), we obtain (26):
p^ X h i
m
1 - p^ X h
pi = N - 1
j
/ c 1 -p^pX^hX h m
j=0
c

(26)

Based on the configuration in (23), (22) can be rewritten as (27):
p^ Z h =

N-1

/

si pi

(27)

i = N/2

and by substituting (26) in (27), we obtain (28):
p^ X h i
m
p^ X h
1
i = N/2
p^ Z h = N - 1
j
/ c 1 -p^pX^hX h m
j=0
N-1

/

c

p ^ X h N/2
p^ X h N
m -c
m
1 - p^ X h
1 - p^ X h
=
N
p^ X h
m
1- c
1 - p^ X h
N/2
p^ X h
c
m
1 - p^ X h
=
p ^ X h N/2
m
1+ c
1 - p^ X h
c

(28)

Adopting the bipolar coding format, which means that
the ranges of real numbers x and z are extended to
-1 # a / x, z # 1, and assuming that the probability of
a bit in the stream being one P ^a = 1 h = ^a + 1 h /2, (28)
leads to (29):
1 + x N/2
` 1- x j -1
z=
1 + x N/2
` 1- x j +1

(29)

By using the Taylor expansion shown in (30), (29) is rewritten to compute tanh based on SC (31):
1 + x . e x ; 1 - x . e -x
N

N

x
- x
2
2
z = e N - e N = tanh ` N x j
- x
x
2
e2 +e 2

and Lemma 3 is proved.
FIRST QUARTER 2021

(30)
(31)
Y

The stochastic implementation of other functions,
such as the absolute value, the exponentiation, and the
max/min functions can be carried out using linear FSMs
[63] and [61]. These functions support, for example,
the development of deep learning applications, as described in Section IV-A. The characteristics of SC make
it suitable, in particular, for designing embedded systems. It offers a lower circuit area and power consumption, as well as high error resilience, in comparison to
their binary system counterparts [64].
Since operations at the logic level are performed on
randomized values, an attractive feature of SC is the degree of tolerance to errors [65]. A stochastic bitstream
of length m represents numbers with a precision of
1/m. The error follows a normal distribution with zero
mean and a variance that is inversely proportional to
the bitstream length N ^v . 1/ N h [56]. For  example,
insufficient bitstream length or randomness can be
leveraged in approximate computing [66], [67].
Several processors, more or less dedicated, have
been developed to apply SC to different applications,
such as Artificial Neural Networks (ANNs), control
systems, and the decoding of modern error-correcting
codes [67]. Stochastic Recognition and Mining processor (StoRM) uses SC to efficiently materialize computational kernels to perform classification, search, and
inference on real data [68]. The StoRM architecture is a
2D array of stochastic Processing Elements (PEs) (typically 15 # 15) with a streaming memory hierarchy, mitigating the overhead of binary-to-stochastic conversion
by sharing the corresponding arithmetic units across
rows or columns of stochastic PEs. Experimental results
from the implementation of StoRM in CMOS TSMC 65 nm
technology show that SC allows a highly compact implementation, resulting in order-of-magnitude less circuit
area and power consumption.
However, computation with SC requires multiple cycles,
equal to the number of bits in the stochastic bitstream.
Since the length of the bitstream grows with the precision
of the data being represented, regardless of its lower power consumption, the PE ends up consuming more energy
than a conventional binary processing element. To mitigate this intrinsic limitation (increase in cycle count) of
SC, vector processing and segmented stochastic processing are adopted [68]. Segmented stochastic processing is
a hybrid representation that divides the binary data into
equally sized segments that are represented by individual
IEEE CIRCUITS AND SYSTEMS MAGAZINE

19
IEEE Circuits and Systems Magazine - Q1 2021

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q1 2021
