Computational Intelligence - May 2016 - 67

between layers. Once pyramidal cells are
triggered to fire, cooperation of ADP and
theta oscillation results in the repetitive
firing of neurons coding for memory
items as short-term memory. Meanwhile,
fast STDP and slow STDP contribute to
the enhancement of intra- and interassembly connections, respectively.
Theta oscillation has been recorded
in hippocampus involved in memory
function. The model proposed in [40]
suggests that memor ies might be
encoded and recalled during different
portions of the theta cycle. Similar
scheme is employed in our model, hetero-associative memory storage occurs
in troughs of theta oscillation, while
stored memories are retrieved in portions near the maximums of theta oscillation. Since the activation of neural
activities at troughs requires strong excitation, the resulted synaptic efficacies are
stronger than required at the maximums. Redundant excitation and distributed information over neurons
improve the robustness of recalling hetero-associative memories. Environmental noises and even information loss will
not lead to a severe retrieval failure as
demonstrated by the simulation results.
Moreover, multiple patterns are encoded
and stored into associative and episodic
memories following a hierarchical organizing principle. Hereto-associative
memory is encoded by the connectivity
between layers. Along with the development of neural assemblies, lateral
connections are enhanced by STDP.
Intra-assembly connections represent
auto-associative memory, while episodic
memory about the sequence of input
patterns is encoded in the form of interassembly connections.
C. temporal Compression
and information Binding

The discovery of place cells suggests
that spatial information can be encoded
by the cellular activities of hippocampus. Moreover, dual oscillations have
been observed to be involved in memory function. In the STM model, memory items are coded by neural assemblies
firing within different gamma cycles,
while past and present events are

Auto-associative memories are formed by intraassembly connections in Layer I, while episodic
memories are formed by inter-assembly connections
in Layer II.
chunked by the theta oscillation. When
presented patterns that have been
learned before, neural assemblies coding
for each of them will be activated correspondingly. The temporal compression
of neural firing volleys contributes
to the generation of inter-assembly
connections and ability to predict upcoming patterns.
Since neural responses in Layer II are
linked with inter-assembly connections,
information about different stimuli is
binded as shown in Fig. 7(c). Temporally
compressed neural patterns can be treated as a new spatio-temporal pattern. By
duplicating this basic network into a
larger network, more powerful ability to
organize neural activities representing
features with different specificity along
the hierarchy can be achieved. Each
basic network binds several patterns
(features) into a combined pattern (feature) and transmits it to a higher level
network as its input stimulus. As a result,
neural activities represent more specific
and complex patterns along the hierarchical network.
D. related Work

The STM model shares some similar
ideas with several existing studies in the
field of neuroscience. The proposed
model simulates neural assembly activities reported in [14], and is in agreement
with the separation of encoding and
retrieval theory suggested by [40], [41].
The mechanism sustaining short-term
memory was used in Jensen et al.'s
model, while tempotron learning and
STDP learning have been employed in
contributing long-term memory formation in other models.
Recurrent networks have been an
important paradigm to implement autoassociative memory [22]. It has been
demonstrated that simultaneous firings
of a group of neurons can be stored in a
fixed recurrent network modeling

hippocampual CA3 area [42]. The idea
that dual oscillation interacts with pyramidal cells has been implemented in the
model. Although firing times of spikes
are considered in the model, the
external inputs exciting a specific pyramidal cell are presumed fire in synchrony, which ignores sensory encoding as
well as the hetero-association process. In
addition, recurrent subnetworks are predefined in the model and input patterns
are presented to specific recurrent networks. These assumptions restrict the
generalization and adaptability of the
STM model.
A sequence learning model based on
short-term memory mechanism was
proposed in [43], which possesses similar
features of recurrent network and hierarchical structure as our model. However, our model implements spiking
neurons based computing to achieve a
more biologically plausible memory
model. Another sequences learning
model in a hierarchical structure proposed by [44] employs prediction mechanism and minicolumn structure to
realize episodic memory. The prediction
mechanism might be considered to
improve our model in the future.
The hierarchical temporal memory
(HTM) [25] aims to develop a machine
learning technology by mimicking the
structural and algorithmic properties of
the neocortex, featured by a sophisticated columns-based structure. In contrast
to fixed structures, our STM model can
generate neural assemblies during the
learning process. The plasticity of network structure not only provides generalization and scale-up capability, but fully
exploits available coding units of the
network. Moreover, the HTM assumes
that neural information is represented by
rate codes and leaves out complexity
and processing power of biological neurons. By using spiking neurons and
incorporating biologically plausible

may 2016 | IEEE ComputatIonal IntEllIgEnCE magazInE

Table of Contents for the Digital Edition of Computational Intelligence - May 2016