IEEE Circuits and Systems Magazine - Q2 2022 - 17

not suitable for neuromorphic computing because it cannot
provide the massive concurrency and parallelism featured
in the biological neural systems.
Ideal Architecture In a biological system, each neuron
and synapse has its own state, which can be characterized
by a set of parameters and variables in software/
algorithm models. These parameters/variable are not
shared among different neurons or synapses, and they are
updated locally and concurrently. The inter-neuron communication
is also a parallel process through massive
number of synapses. Based on above observations, an
ideal architecture should support 1) Local and dedicated
data storage; 2) Massive concurrency; and 3) High connection
density.
Based on above requirements, an ideal architecture
of digital neuromorphic hardware is presented in Figure
9(b) where each processing unit and its local memory
are used to represent a single neuron, and the local
Arithmetic/logic unit (ALU) is responsible for updating
neuron status. The close-to-memory computing reduces
data retrieving latency, while the distributed memory enables
parallel computation. This ideal architecture maximizes
the number of synaptic operations per second
(SynOps/s), which is an important measure of neuromorphic
hardware performance.
Practical Architecture The aforementioned ideal neuromorphic
architecture, that maintains one processing
unit for each neuron, is not scalable when the size of the
neural network increases. The large circuit overhead arising
from an ALU assigned to each neuron and hardwiring
the neurons to each other for large-scale SNNs is highly
impractical. A practical solution is to group a number of
neurons in a core, as shown in Figure 9(c). These neurons
have their own local data, but share the same data path to
update neuron and synapse status. Compared to the ideal
neuromorphic architecture, this reduces the effective circuit
area per neuron significantly. The cores also enable
sharing of common parameters among the neurons for
a more efficient usage of memory. However, as the same
ALU is utilized to update the neurons and synapse status
associated with a core, usually time-multiplexing is utilized.
This reduces the parallelism, as the ALU can only be
accessed by one neuron at a time. Additionally, this also
introduces spiking delays (or delay in neuron update)
that can cause errors in the neuronal encoding. Trade-offs
of above three architectures are shown in Table I.
D. Communication
Neuromorphic systems support both intra-chip and interchip
communication. Both types of communications are
implemented using address event representation (AER).
[226]-[228] apply AER to on-chip inter-neuron packet
based communications. Vainbrand and Ginosaur studSECOND
QUARTER 2022
ied different network-on-chip architectures for neural
networks, including mesh, shared bus, tree, and point-topoint.
They found network-on-chip multicast to have the
highest performance [229]. Ring-based communication
structure has been tested successfully [230], [231] for
on-chip neuron communications. Buses were also utilized
for some on-chip communication systems [232], [233].
This asynchronous bundled data design style is well suited
for SNNs that fundamentally feature a high degree of
sparseness in their activity across both space and time.
[234]-[239] also apply AER to inter-chip communications,
where the chip ID is encoded as part of the packet address.
AER have been implemented through custom PCI
boards to optimize performance [240], [241] or utilizing
FPGAs [242]-[244].
E. Supporting Software and Ecosystems
Supporting software tools are important components in
the ecosystem of neuromorphic processors. Those usually
consists of tools for mapping, programming and simulation.
The mapping tools partition an SNN into clusters
and map clusters to processing units on the neuromorphic
hardware [132], [245]-[250]. The goal is to minimize
the inter-core communication that considers the hardware
constraints such as the number of input/output
channels, the amount of local memories, etc. Programming
tools enable users to explicitly describe a particular
neuromorphic architecture [251]-[255] by setting different
parameters and topology configurations, or by utilizing
custom training methods. Software simulators [131],
[249], [253], [256]-[259] are used to emulate the neuromorphic
hardware and enable the user base in developing
and testing of network topologies, training algorithms,
neuron parameters, etc., when the hardware has not been
widely deployed.
IV. Case Studies of some Large-Scale
Neuromorphic Systems
In this section, we will discuss several representative
systems as examples to showcase the main components
discussed in previous section. They are: neuromorphic
super computing platform (SpiNNaker), digital ASIC
(TrueNorth), digital ASIC with on-chip learning (Loihi),
analog and mixed-signal design (BrainScaleS), and ANNSNN
hybrid design (Tianjic). A quick summary of the
Table I.
Trade-Offs of Different Architectures.
Architecture
Von Neumann
Ideal
Practical
Scalability
++
-
++
Cost
performance
++ -
-
+
+++
++
IEEE CIRCUITS AND SYSTEMS MAGAZINE
17
IEEE Circuits and Systems Magazine - Q2 2022

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q2 2022
