IEEE Computational Intelligence Magazine - August 2019 - 53

indefinite error function through the
ZND design formula. On the basis of the
ZND methodology, various continuoustime ZND models can be obtained to
solve different time-varying problems.
Notably, in terms of practical applications (e.g., hardware implementation
and numerical computation), discretetime models are the basic consideration
for common digital equipment. Therefore, the continuous-time model directly
obtained by the ZND methodology is
supposed to be converted into a discrete-time model, and a discretization
formula is needed for the conversion.
However, almost all the traditional
numerical differentiation formulas cannot be applied to obtain discrete-time
ZND models due to some strict conditions that they should conform to. The
discretization formulas should be 1-stepahead because the discrete-time models
obtained are unable to exploit the information of future time instants in real-time
applications. Additionally, the discrete-time
models should have 0-stability, whereas
almost all the 1-step-ahead numerical differentiation formulas in the existing literature are unable to stabilize the resultant
models [16], [17].
To the authors' best knowledge, the
Euler forward-difference formula was
the only and simplest numerical differentiation formula that can be adopted
for time discretization in the past [18].
However, the Euler forward-difference
formula is not very precise, which deeply
affects the performance of the discretetime models. In recent years, Zhang et al.
discretization (ZeaD) formulas have been
proposed to obtain the discrete-time
models with high precision, and these
have successfully discretized continuoustime ZND models.Various discrete-time
models are obtained by using ZeaD formulas. The current study aims to introduce the ZND methodology, present
readers with new discretization formulas
and various discrete-time ZND models,
and point out future research directions
with the following contents.
1) The relationship between ZND
models and Zhao-Lu-Swamy-Feng
(ZLSF) models [19]-[21], as proposed
by Zhao, Lu, Swamy and Feng, is dis-

covered; i.e., ZLSF models are minimization-type and Euler-type special
cases of ZND models.
2) The strict conditions for the discretization of continuous-time ZND models are briefly discussed, and the models,
which are discretized by different discretization formulas, are compared.
3) The performance and factors related
to the convergence of different discrete-time ZND models are analyzed and discussed.
4) Discrete-time ZND models are
compared with discrete-time derivative dynamics (DD), gradient trajectory tracking (GTT), and Newton
trajectory tracking (NTT) models
for time-varying minimization
problem solving [3], [4], [22]. The
differences between ZND and other
approaches are demonstrated.
II. ZND Design and ContinuousTime Models

ZND is a special class of neural dynamics
designed for real-time solutions of timevarying problems. Moreover, ZND models
are established on the basis of the elimination of an indefinite error function. For
example, let us consider solving a timevarying equation problem, i.e., at any time
instant t ! [0, +3), finding out x(t ) ! R
that makes the following equation true:
f (x(t ), t ) = 0.

(1)

The existence of the theoretical timevarying solution x )(t ) in Eq. (1) is guaranteed, which is the precondition for our
discussion. We start by defining an indefinite error function e(t ) to solve this
problem.The definition of error function
e(t ) depends on the specific type of the
problem encountered, and e(t ) may
sometimes have various feasible forms. In
this problem, the extent of deviation
from f (x(t ), t ) to zero should be quantified by the error function. The error
function can simply be given by [23]:
e(t ) = f (x(t ), t ).

(2)

The ZND design formula is adopted to
make e(t ) converge to zero rapidly,
which is given by [13]-[15], [23]-[25]:
eo (t ) =-pz(e(t )),

(3)

where p ! R + is a design parameter,
and z($) represents a monotonicallyincreasing odd activation function. Notably, the choice of activation functions has
a significant effect on the performance of
the corresponding ZND models. The
convergence of the error function is
related to the activation function, and the
relevant problems have been discussed/
investigated in previous studies [15], [26],
[27]. Specifically, when z($) is a simple
linear function, i.e., z(x) = x, formula (3)
can be simplified as [13], [23]:
eo (t ) =-pe(t ).

(4)

The theoretical solution of differential
Eq. (4) is e(t ) = e(0) exp(-pt ), which
ensures that error function (2) converges
to zero exponentially. According to the
definition of the error function (2) and
Eq. (4), we have the following continuous-time ZND model [23]:
2f (x(t ), t )
2f (x(t ), t )
xo (t ) =-pf (x(t ), t ) .
2x
2t
The time-varying problem of Eq. (1)
is a representative of many time-varying
problems, and its solving process reflects
the essence of the ZND methodology.
Different time-varying problems, such as
time-varying root finding, linear/nonlinear
equation solving, minimization and matrix
inversion, are solved by this methodology [13]-[15], [23]-[25]. In addition to
Table  I of [8], in this article, Table  I
shows some continuous-time ZND
models corresponding to different timevarying problems1 [13]-[15], [23]. In
Table  I, x(t ) = [x 1(t ), x 2(t ), f, x n(t )]T,
f ( x (t ), t ) = [ f1 ( x (t ), t ), f 2 ( x (t ), t ), f,
fn ( x (t ), t )]T, and U($) denotes a vector
array whose elements are monotonically-increasing odd activation functions.
For better understanding, let us discuss the time-varying minimization
problem to observe the manner by
which the ZND methodology is applied
to solve a specific time-varying issue.The
continuous-time minimization problem
can be stated as follows [28], [29]:
1
Supplementary materials are provided at http://www
.qizhy.com/Materials/CIM-ZND/SupplementaryMaterials.pdf, which includes the tables of other ZND
models and the models for numerical experiments.

AUGUST 2019 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE

53
http://www.qizhy.com/Materials/CIM-ZND/Supplementary-Materials.pdf http://www.qizhy.com/Materials/CIM-ZND/Supplementary-Materials.pdf http://www.qizhy.com/Materials/CIM-ZND/Supplementary-Materials.pdf
IEEE Computational Intelligence Magazine - August 2019

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - August 2019
