IEEE Circuits and Systems Magazine - Q2 2022 - 21

where []
xt is the trace variable and []
St is the incoming
spike train. The traces are used by the learning engine as
input variables for synaptic adaptation. Loihi supports
in-hardware adaptation for all three synaptic variables,
weight, delay and tag. The locality constraint is satisfied
during the procedure. The weight (delay, tag) can only
be accessed and modified by the postsynaptic neuron,
based only on locally available information, such as the
spike trains from the presynaptic (source) and postsynaptic
(destination) neurons. The functional form of adaptation
rules is described in sum-of-products form in terms
of microcode operations associated with the synapse:
Np
zz ()
1
=+ /
i==1
Ax C,,iijij
j
%
+
ni
(19)
where z is the transformed synaptic variable (weight, delay
or tag), x ,ij
able to the learning engine, and C ,ij and Ai
refers to some selected input traces availare
microcodespecified
signed constants [105]. Based on Eq. 19, the
learning engine supports simple pairwise STDP rules and
also much more complicated rules such as triplet STDP
[265], [266], reinforcement learning with synaptic tag assignments
[267], and complex rules that reference both
rate averaged and spike-timing traces.
2) Architecture and Implementation
The Loihi processor is a digital and functionally deterministic
neuromorphic chip. It was implemented in an
asynchronous bundled data design style allowing for
event-driven communication through spikes with maximal
activity gating during idle periods. It was fabricated
in Intel's 14nm FinFET process. The chip has a die area
of 60 mm2 containing 2.07 billion transistors and consists
of 128 neuromorphic cores and three x86 cores. Loihi includes
a total of 16MB of synaptic memory. It boasts a
maximum synaptic density of 2.1 million unique synaptic
variables per mm2 with its densest 1-bit synapse format
and maximum neuron density of 2,184 per mm2. After adjusting
for the benefit from the advanced technology, this
comes to a 2× reduction in the neuron density in comparison
to TrueNorth, which can be interpreted as the cost of
Loihi's greatly expanded feature set.
Each neuromorphic core in Loihi implements 1,024
primitive spiking neural units called compartments,
which can be grouped into sets of trees constituting neurons.
The architecture memory for the storage of configuration
and state variables for the compartments and the
associated connectivity (fan-in and fan-out) are shared in
a core. Every algorithmic timestep, the state variables are
updated in a time-multiplexed, pipelined manner. When a
neuron's activation exceeds the threshold, it generates a
spike message that is routed to the fan-out compartment
in one or multiple destination cores.
SECOND QUARTER 2022
An asynchronous network-on-chip (NoC) forms the
backbone for communication between the cores. All communication
between cores occurs in the form of packetized
messages. The different types of communication messages
include core management and x86-to-x86 messaging, spike
messages, and barrier messages for time synchronization
between cores. The NoC distributes the communication
messages according to the dimension-order routing. The
NoC itself only supports unicast distributions. To multicast
spikes, the output process of each core iterates over a list
of destination cores for a firing neuron's fan-out connections
and sends one spike per core.
The host CPU, the on-chip x86 processors and the
neural cores can communicate with each other using
any type of messages. For off-chip communication over a
second-level network, messages may be hierarchically encapsulated.
The mesh protocol allows for scaling to 4096
on-chip cores and up to 16,384 chips.
3) Supporting Software/Software Ecosystem
Loihi provides a Python-based API that can be used to
specify complex SNN topologies and to program custom
learning rules. It also provides a compiler and runtime library
for building and executing SNNs on Loihi. The API utilizes
core primitives: neuronal compartments and synaptic
connections as means of defining SNN topology, synaptic
traces and a neuron model to describe SNN dynamics, and
synaptic learning rules. Thus, enabling the programmers
to implement SNNs in an intuitive way without requiring
intimate knowledge of its architectural details. The compiler
takes an SNN implementation and produces a binary
byte stream in three steps: preprocessing, resource allocation,
and code generation. Due to the support of more
complex neuron and synapse models, the application of
Loihi is more diversified. It has been applied to accelerate
the process of Locally Competitive Algorithm for LASSO
[105], Neural Engineering Framework (NEF) [52], Stochastic
SNNs for solving Constraint Satisfaction Problems [268],
Parallel graph search [269] and Random diffusion walkers
[270]. It has also been used to implement biological inspired
systems such as Olfaction-inspired rapid learning
[271]. Dynamic Neural Fields [272], SLAM [72], Evolutionary
search [273] are fields to which Loihi is being applied.
It has also been used to implement deep SNN for conventional
machine learning applications such as classification
or prediction. For these applications, Nengo is used to convert
a DNN to SNN [274], [275].
C. Analog/Mixed-signal System: BrainScaleS
Analog/mixed-signal design has always played an important
role in neuromorphic computing due to its analogue
to biological systems. After modeling the neurons
and synapses using circuits consisting of resistors, and
IEEE CIRCUITS AND SYSTEMS MAGAZINE
21
IEEE Circuits and Systems Magazine - Q2 2022

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q2 2022
