IEEE Computational Intelligence Magazine - August 2021 - 54

The neural architecture of DNR consists of four layers,
namely the synaptic layer, dendritic layer, membrane
layer, and soma layer.
neurons. These elements are distributed throughout the dendritic
tree and possess various receptors for specific ions.
Depending on the potential of the ions entering a receptor,
the synapse changes its connection state and enters either an
excitatory or inhibitory state [45]. The process of signal
transmission can be described using the following equation:
1
y =im ()
1 e+ -where
xi
[0,1], with ! 612 f ,, ,;
kw xqim iim
,
(1)
represents the i-th input feature, whose range is
@ yim
iI is the output of the i-th syn!
612 f ,, ,;
and
DNM. Therefore, in DNR, each um
is
regarded as a parameter that needs to be
optimized via a learning algorithm; accordingly,
these values are specified differently
for different problems.
D. Cell Body (Soma)
The soma fires depending on whether the membrane potential
exceeds a given threshold. This process can be mathematically
described as a sigmoid operation on the product terms,
as follows:
O =
1
1 e ()
+ -kV
qs
,
(4)
where V represents the output of the membrane layer and k
and qs are positive constant hyperparameters.
apse on the m-th dendritic branch, withmM@ k
is a hyperparameter that is a positive constant; and wim
qim are connection parameters that represent a weight and a
threshold, respectively. To obtain the appropriate values for
each problem, these connection parameters in a DNR model
can be trained using a learning algorithm.
B. Dendrite Layer
Each branch in the dendrite layer receives the output signals
from all synapses on that branch. The nonlinear relationship
among these signals plays a key role in neural information
processing for several sensory systems in biological networks,
such as the visual and auditory systems [46], [47]; in DNR,
this relationship can be expressed in terms of multiplication
operations. Let Zm
represent the output of the m-th dendritic
branch. The equation for a dendritic branch can be expressed
as follows:
I
Zy .
i =1
m = % im
(2)
C. Membrane Layer
The membrane layer combines all outputs from the dendrite
layer through a summation operation. Let V represent the output
of the membrane layer. The corresponding equation can
be expressed as follows:
M
Vu ,Zmm
m
= /^h
=1
)
where um
(3)
represents the strength of the m-th dendritic
branch. This value is constant and is always set to 1 for each
branch in the original DNM to simplify the neural architecture
to accelerate the computation process [29]. However, in
reality, the thicknesses and signal transformation strengths of
the dendritic branches vary; thus, using a uniform um
for all branches may degrade the regression ability of the
value
54 IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | AUGUST 2021
E. Connection Cases
On the right side of Fig. 1, the six functions of the synaptic
layer are illustrated for various combinations of wim
and q .im
According to these six different functions, the connection
states of the synaptic layer can be divided into four main categories,
defined as follows: constant 1 connections, whose
parameters satisfy wq0im
11 or 11 implying
im
0 wq ,
im
im
that regardless of the input, the output is always excitatory;
constant 0 connections, whose parameters satisfy qw 0
or qw ,im11 implying that regardless of the input, the
im11
im
im 0
output remains inhibitory; excitatory connections, whose
parameters satisfy
im
im
im
input and output are inversely correlated.
F. Learning Algorithm
Because of the multiplication operations applied in the dendrite
layer, the parameter space of the model appears to be
extremely large and complicated. Additionally, weights are
added to the output of each dendritic branch in the DNR
model, leading to an increase in the dimensionality of the
parameter space, which further increases the difficulty of
optimizing the parameters. Consequently, it is difficult to
perfectly train the parameters when using the traditional BP
algorithm. Therefore, in this study, the SMS algorithm is
adopted as a more suitable global optimization algorithm to
optimize the DNR model. The SMS algorithm is an evolutionary
algorithm that mimics the variation in the states of
matter. Compared with the traditional BP algorithm, the
SMS algorithm exhibits a higher search ability, is less likely to
fall into local optima, and seldom results in overfitting. In this
subsection, this algorithm is described in detail.
The process of searching for the best solution in the SMS
algorithm can be expressed as a series of physical motions
among molecules, which mimic the state transformations of
matter [38]. Specifically, the SMS algorithm can be divided
0 qw ,im11 implying that the input and
output are directly correlated; and inhibitory connections,
whose parameters satisfy
wq ,011 implying that the
IEEE Computational Intelligence Magazine - August 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - August 2021
