IEEE Circuits and Systems Magazine - Q2 2022 - 7

parallel and distributed computing inspired by biological
neural systems, namely neuromorphic computing. By
learning from the biological and physical characteristics
of the neocortex system, researchers in neuromorphic
computing incorporate a brain-inspired computing model,
a non-conventional architecture, and novel device technology
to provide energy efficient solutions to real-life machine
intelligence problems.
The concept of neuromorphic computing was first
proposed by Carver Mead in the 1980s [1]-[4]. The early
works in this area focused on emulating the analog behavior
of neural systems. It is observed that biological systems
achieve many orders of magnitude higher efficiency than
digital systems when performing certain cognitive tasks.
[1] and [4] credit such advantage to the fundamental differences
between digital circuits and biological systems. The
early works in neuromorphic computing tried to bridge the
gap between the lower-level physical details of biological
systems and the higher-level computational functionality.
[2], [5] claim that, due to their adaptability, neuromorphic
systems are more resilient to noise and component failure
and have the potential to be more energy efficient.
The early efforts of neuromorphic computing include
[1]-[4], [6]-[9]. Those works mainly focus on modeling
realistic biological systems using analog circuits. [7] developed
a silicon retina and a sensorimotor system. [8]
designed an electronic cochlea using CMOS which shares
the same principle as biological cochlea. [1] proposed a
chip that is structurally similar to retinas of higher animals.
[10] developed a floating-gate silicon MOS transistor
to emulate synapse and realized a learning rule on the
synapse array.
The implementation of neuromorphic computing has
shifted to the digital domain in recent decades for better
noise resilience and higher scalability. The research focus
has also extended from single neuron implementation to
network and inter-neuron communication architectures.
In addition to digital systems, emerging materials and
devices such as memristors, phase changing materials,
photonic circuits are also being investigated for hybrid
solutions of neuromorphic computing. Spiking neural network
(SNN) is often studied together with neuromorphic
computing as the underlying computational model. Sometimes
the two terms are even interchangeable. SNNs have
more biologically plausible features than conventional
artificial neural networks (ANNs) [11]. Similar to the biological
neural system, SNN is inherently a dynamic and
stateful network. The most distinct property of SNN is
that the information is represented, transmitted and processed
as discrete spike events, also referred to as action
potentials [12]. Spikes are electrical pulses in biological
neural systems. In SNN mathematical models, spikes are
usually represented by Dirac Delta functions. Although a
spike enables low power information transmission and
processing, the non-differentiable Dirac Delta function
also imposes a major challenge in SNN training, hindering
the application of gradient descent algorithms [13]. In
addition, unlike ANN, in which inter-neuron connections
pass information lossless with a linear scaling controlled
by the weight coefficients, connections/synapses of SNN
may consist of multiple state variables and parameters.
This feature makes the SNN more powerful in processing
spatial/temporal sequences, but also increases the complexity
of its implementation.
It is noteworthy that the boundary between SNN and
ANN is not always clear. Though most SNN models use
spikes, there are also rate-based SNN models, in which
the output of a neuron is no longer discrete spikes, but
real-valued instantaneous spike rates. Such models can
be interpreted as ANNs [14]. There are also models [15],
[16] and hardware [17] that fuse SNN and ANN together.
In this work, the name SNN is used to refer to the models
that generate spikes as their outputs.
While the inferencing and learning of conventional
ANNs are generally formulated as matrix-vector multiplications,
there is no unified model for SNNs. Different
models for spiking neurons and synapses represent their
biological counterpart at different levels of details, which
impacts the flexibility, complexity, and efficiency of hardware/software
implementations. Based on their applications,
we can divide neuromorphic computing into two
categories, systems for computational neuroscience and
systems for machine intelligence. Although their boundary
is not always clear, the former usually focuses on
models with more biophysical details and tries to reproduce
their physiological features such as network oscillations.
The latter focuses more on mathematically abstract
models and their information representation and retrieval
abilities. In Figure 1, we divide neuromorphic computing
systems into 8 main categories based on their computational
model, implementation, and applications. In this
paper we will limit ourselves to the digital or mixed signal
implementation of spiking neural networks for machine
intelligence applications. Compared to earlier survey
[18], which comprehensively discusses various aspects
of neuromorphic computing, including history, model, algorithm,
hardware design, device and applications, this
work focuses more on the algorithm-hardware codesign.
Amar Shrestha, Haowen Fang, Zaidao Mei, Daniel Patrick Rider, and Qinru Qiu are with the Department of Electrical Engineering and Computer Science,
Syracuse University, Syracuse, NY (e-mails: {amshrest, hfang02, zmei05, dprider, qiqiu}@syr.edu). and Qing Wu is with the US Air Force Research
Laboratory, Rome, NY, USA (e-mail: qing.wu.2@us.af.mil).
SECOND QUARTER 2022
IEEE CIRCUITS AND SYSTEMS MAGAZINE
7

IEEE Circuits and Systems Magazine - Q2 2022

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q2 2022