IEEE Circuits and Systems Magazine - Q2 2018 - 3

From
the
theEditor
Editor
Chai Wah Wu
Editor-in-Chief, IEEE Circuits and Systems Magazine

t is my immense pleasure to introduce this special issue edited by Prof. Ronald Tetzlaff and Prof. Dalibor
Biolek in honor of Prof. Leon Chua's 80th birthday.
Leon Chua was much more than just my Ph.D. dissertation supervisor. He was also my mentor, friend, colleague, advisor, and role model. He has a knack for finding beauty, elegance, and excitement in the sometimes
mundane engineering topics. While his accomplishments
in areas such as nonlinear circuit theory [1] advance the
current state of the art and are found in textbooks and
course curricula, many of his discoveries have continued
to find important novel applications many years later.
The discovery of memristors as the "missing" circuit element, described in his landmark 1971 IEEE TCAS paper
[2], was an intellectual tour de force. The traditional circuit elements of resistors, capacitors and inductors were
known for over a century, but Leon saw them as part of
the systematic relationships between the fundamental
properties of charge, voltage, time, and flux and deduce
the missing part in these relationships. Only recently has
it been practical to manufacture memristors at scale in
novel memory architectures and this is now a thriving
research effort. By the time I joined Leon's lab, Leon has
changed his focus to study nonlinear dynamics, in particular chaotic dynamics of nonlinear circuits. This has
been observed since Van der Pol [3] at the beginning of
the last century, but only beginning to be understood in
the last few decades. The introduction of Chua's circuit
[4] provides a useful theoretical and practical device for
studying such chaotic phenomena, especially in electronic circuits. The beautiful pictures of the myriad of
strange attractors belie the immense complexity within.
He also introduced the world to Cellular Neural Networks [5], a powerful paradigm of analog computing that
uses shift invariant templates to reduce the complexity
of the network and leverage the shift invariant nature of
many computer vision tasks such as edge detection. The
feedback template allows computation to be iterated via
the continuous-time dynamics and this has resulted in
many nontrivial but useful dynamics [6], including chaos, whereas the feedforward template allows partial re-

Digital Object Identifier 10.1109/MCAS.2018.2821720
Date of publication: 21 May 2018

second quarter 2018

With diana and Leon chua at IWcsn shanghai, 2014.

sults to be propagated to the next layer. The technique
of using shift invariance to reduce the number of free parameters was also found useful in feed forward Convolutional Neural Networks (same acronym!) to successfully
tackle various difficult computer vision problems [7].
One of Cellular Neural Networks' strengths is that the
use of feedback template allows the complex dynamics
of the resulting ODE to play a role in the computation.
Recently, Convolutional Neural Networks coupled with
Residual Networks are analyzed as the Forward Euler
discretization of an ODE that is of the same form as a
Cellular Neural Network [8].
Leon also conducted fundamental research in piecewise-linear (PWL) functions [9]-[11] and introduced
Canonical Piecewise-Linear representations of multidimensional PWL functions. PWL activation functions were used in the original formulation of Cellular
Neural Networks and recently is becoming more important in deep learning as PWL activation functions such
as rectified linear units (ReLu) were found to be easier
to implement and improve training speed as they overcome the "vanishing gradient" problem when sigmoid
functions are used in back propagation training. A recent paper on deep learning neural networks with ReLu
Ieee cIrcuIts and systems magazIne

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q2 2018