IEEE Circuits and Systems Magazine - Q2 2022 - 15

building blocks, whose implementations are designed
first. These building blocks are connected by a communication
network to form a system architecture. Different
implementations and hardware architectures are selected
for neuron and synapse models with different degrees of
complexity. The leaky integrate and fire (LIF) model in
Equation 3 has been a popular choice for hardware implementation
[105], [112]-[114] since it is simple but still retains
some temporal dynamics. Complex neuron and synapse
behavior specified in Equation 9 requires specific
hardware to efficiently compute their evolution through
time. The LIF can be reduced into an IF model, which can
be implemented cost effectively using an adder, a comparator,
and a memory [11], for input integration, threshold
detection and membrane potential storage, respectively.
To achieve higher degree of fidelity to biological models,
ionic channels and other bio-realistic components
have been implemented [113], [115], [116]. [117] implements
advanced reconfigurable units based on the work
of Izhikevich [26] or bio-realistic ion channels [116] interaction
in fully digital designs. The SpiNNaker [118] can be
used to evaluate detailed biological neuron and synapse
models at a high computation cost. These implementations
of highly bio-plausible neurons and synapses provide
insights of the brain function from the neuroscience
point of view. They usually are not used for machine intelligent
applications.
B. Implementation Choices
Based on their implementation choices, neuromorphic
systems can be categorized into three categories, (1) digital,
(2) analog, or (3) mixed signal platform.
Digital neuromorphic systems can further be divided
into CPU based, Application Specific Integrated Circuit
(ASIC) based and FPGA based implementations. An example
of CPU based implementation is SpiNNaker. SpiNNaker
is an ARM based, fully digital massively parallel
system. It is composed of thousands of ARM cores and
a custom interconnect communication scheme optimized
for spike-based network communication. The processing
unit itself is general purpose and not customized for neuromorphic
functions [40], [118]-[133].
IBM's TrueNorth [114], [117], [134]-[139] and Intel's
Loihi [105] are well known examples of fully custom ASIC
implementation of neuromorphic systems. Some other
examples of ASIC based neuromorphic systems include
[63], [64], [140]-[155].
Most ASICs are subject to limitations of specific neuron
models and algorithms. Therefore, FPGA has also drawn
much attention for its flexibility. FPGA has been widely
used for exploring various aspects of neuromorphic hardware
and algorithms research. Most of these works adopt
a multi-core architecture [34], [156]-[159]. Due to the limSECOND
QUARTER 2022
ited resource of a single FPGA, there are also works utilizing
multiple FPGAs [160]-[162]. FPGA's flexibility also
lends it for exploration into various in-hardware training
algorithms. Some examples are: a modified STDP rule that
uses shift operation to replace the exponential operation
to reduce logic resource consumption [157]; competitive
Hebbian learning on chip with biologically plausible
Izhikevich neurons on FPGA [163]; a hardware friendly
STDP rule which allows low bit precision in a liquid state
machine (LSM) on FPGA [164]; STDP for convolutional
SNN on FPGA [165]; and an STDP rule that uses only 1-bit
synaptic weights to reduce computing, communication,
and memory overhead [166].
For different biological neuron behaviors, such as
conservation of charge, amplification, thresholding and
integration, the analog circuit analogies can be found [2].
Such similarity makes analog integrated circuits and neuromorphic
systems well suited for each other. The original
neuromorphic definition by Carver Mead referred to
analog circuits that operated in subthreshold mode [2].
Many analog neuromorphic systems also operate in this
region typically for power efficiency [90], [167]-[176].
There are a large variety of other neuromorphic analog
implementations [90], [177]-[209].
Similar to digital FPGAs, there are field programmable
analog arrays (FPAAs) enabling programmability for
analog neuromorphic systems [210]-[214]. Some custom
FPAAs are developed specifically for neuromorphic
systems, including the field programmable neural array
(FPNA) [215] and the NeuroFPAA [216]. While many of
the digital neuromorphic systems adopt asynchronous
and event driven methods for energy efficiency, analog
neuromorphic systems do sometimes employ clocks for
synchronization.
Mixed analog and digital implementation is usually the
solution to overcome some inherent limitations of analog
implementation. In many analog neuromorphic systems,
synaptic weights are stored in digital memory for reliability
and longer duration [217]-[220]. In some analog neuromorphic
systems, digital communication is utilized either
within the chip, or among neuromorphic chips [221].
These communications are usually in the form of digital
spikes. Using digital components for programmability or
learning mechanisms is also common [222]-[225]. Two
major projects within the mixed analog/digital family are
Neurogrid and BrainScaleS.
C. Architecture
In this section, we discuss three different architecture
choices, their pros and cons, as well as implications on
hardware design. These three choices are von Neumann
architecture, ideal architecture for neuromorphic computing
and practical multi-core architecture.
IEEE CIRCUITS AND SYSTEMS MAGAZINE
15
IEEE Circuits and Systems Magazine - Q2 2022

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q2 2022
