Computational Intelligence - May 2016 - 68
mechanisms, our model is able to simulate the complex spiking neural dynamics for memory for mulation and
organization, which is a distinct feature
compared to other cognitive learning
and memory architectures aiming for
developing machine learning alternatives
(e.g., the hypernetwork model [45]).
Therefore, the STM model provides
a comprehensive approach to build up
low-level neural circuits for neuromorphic computing such as neuromorphic
chips [46]. As brain-inspired approaches
have been applied to solve various realworld problems [47], [48], efficiently
implementing the STM model on platforms such as VLSI can utilize the
in herent advantage of parallelism of
neuromorphic computing.
V. Conclusion
In this paper, the spatio-temporal memory (STM) model was introduced. The
proposed model is able to store and
recall both associative and episodic
memories with a hierarchical structure.
Throughout the STM model, temporal
codes and temporal learning were integrated to process external stimuli and
formulate memory. The results showed
that neural assemblies can serve as the
internal representation of memory.
They also demonstrated that memories
can be stored in the intra- and interassembly connections and organized in
a hierarchical manner in consistent
with neural mechanisms in the brain.
Our model provides a comprehensive
substrate to elucidate the complex process of memory formulation and organization in virtue of complex spiking
neural dynamics. Real-world stimuli
such as visual and auditory signals can
be employed as the sensory information
to investigate potential applications of
STM model. Being able to more faithfully implement the dynamic details of
memory formulation, our model will
provide more insights to the design of
neuromorphic cognitive systems.
References
1] M. Meister and M. J. B. II, "The neural code of the
retina," Neuron, vol. 22, no. 3, pp. 435-450, 1999.
[2] P. Heil, "Auditory cortical onset responses revisited.
I. First-spike timing," J. Neurophysiol., vol. 77, no. 5, pp.
2616-2641, 1997.
68
[3] J. Perez-Orive, O. Mazor, G. C. Turner, S. Cassenaer,
R. I. Wilson, and G. Laurent, "Oscillations and sparsening of odor representations in the mushroom body," Science, vol. 297, no. 5580, pp. 359-365, 2002.
[4] M. R. Mehta, A. K. Lee, and M. A. Wilson, "Role of
experience and oscillations in transforming a rate code into
a temporal code," Nature, vol. 417, pp. 741-746, 2002 .
[5] R. VanRullen, R. Guyonneau, and S. J. Thorpe,
"Spike times make sense," Trends Neurosci., vol. 28, no. 1,
pp. 1-4, 2005.
[6] C. Kayser, M. A. Montemurro, N. K. Logothetis, and
S. Panzeri, "Spike-phase coding boosts and stabilizes information carried by spatial and temporal spike patterns,"
Neuron, vol. 61, no. 4, pp. 597-608, 2009.
[7] R. Natarajan, Q. J. Huys, P. Dayan, and R. S. Zemel,
"Encoding and decoding spikes for dynamic stimuli,"
Neural Computat., vol. 20, no. 9, pp. 2325-2360, 2008.
[8] J. M. Samonds, Z. Zhou, M. R. Bernard, and A. B.
Bonds, "Synchronous activity in cat visual cortex encodes collinear and cocircular contours," J. Neurophysiol.,
vol. 95, no. 4, pp. 2602-2616, 2006.
[9] A. Mizrahi, A. Shalev, and I. Nelken, "Single neuron
and population coding of natural sounds in auditory cortex," Curr. Opin. Neurobiol., vol. 24, pp. 103-110, 2014.
[10] R. Wyss, P. König, and P. F. M. J. Verschure, "Invariant representations of visual patterns in a temporal
population code," Proc. Natl. Acad. Sci. USA, vol. 100,
no. 1, pp. 324-329, 2003.
[11] M. Boerlin and S. Denève, "Spike-based population
coding and working memory," PLoS Computat. Biol.,
vol. 7, no. 2, p. e1001080, 2011.
[12] L. Lin, R. Osan, S. Shoham, W. Jin, W. Zuo, and
J. Z. Tsien, "Identification of network-level coding units
for real-time representation of episodic experiences in
the hippocampus," Proc. Natl. Acad. Sci. USA, vol. 102,
no. 17, pp. 6125-6130, 2005.
[13] R. Kiani, H. Esteky, K. Mirpour, and K. Tanaka,
"Object category structure in response patterns of neuronal population in monkey inferior temporal cortex," J.
Neurophysiol., vol. 97, no. 6, pp. 4296-4309, 2007.
[14] L. Lin, R. Osan, and J. Z. Tsien, "Organizing principles of real-time memory encoding: Neural clique assemblies and universal neural codes," Trends Neurosci.,
vol. 29, no. 1, pp. 48-57, 2006.
[15] M. Tsodyks, "Spike-timing-dependent synaptic
plasticity-The long road towards understanding neuronal
mechanisms of learning and memory," Trends Neurosci.,
vol. 25, no. 12, pp. 599-600, 2002.
[16] B. Szatmáry and E. M. Izhikevich, "Spike-timing
theory of working memory," PLoS Computat. Biol.,
vol. 6, no. 8, p. e1000879, 2010.
[17] R. Gütig and H. Sompolinsky, "The tempotron: A
neuron that learns spike timing-based decisions," Nature
Neurosci., vol. 9, no. 3, pp. 420-428, 2006.
[18] F. Ponulak and A. Kasinski, "Supervised learning in
spiking neural networks with resume: Sequence learning, classification, and spike shifting," Neural Computat.,
vol. 22, no. 2, pp. 467-510, 2010.
[19] R. V. Florian, "The chronotron: A neuron that learns
to fire temporally precise spike patterns," PLoS ONE,
vol. 7, no. 8, p. e40233, 2012.
[20] J. Hu, H. Tang, K. C. Tan, H. Li, and L. Shi, "A
spike-timing-based integrated model for pattern recognition," Neural Computat., vol. 25, no. 2, pp. 450-472, 2013.
[21] Q. Yu, H. Tang, K. C. Tan, and H. Li, "Precisespike-driven synaptic plasticity: Learning hetero-association of spatiotemporal spike patterns," PLoS ONE, vol. 8,
no. 11, p. e78318, 2013.
[22] H. Tang, H. Li, and R. Yan, "Memory dynamics in
attractor networks with saliency weights," Neural Computat., vol. 22, no. 7, pp. 1899-1926, 2010.
[23] E. Y. Cheu, J. Yu, C. H. Tan, and H. Tang, "Synaptic
conditions for auto-associative memory storage and pattern completion in Jensen et al.'s model of hippocampal area
CA3," J. Computat. Neurosci., vol. 33, no. 3, pp. 435-447, 2012.
[24] S. Schrader, M.-O. Gewaltig, U. Körner, and
E. Körner, "Cortext: A columnar model of bottomup and top-down processing in the neocortex," Neural
Netw., vol. 22, no. 8, pp. 1055-1070, 2009.
[25] D. George and J. Hawkins, "Towards a mathematical
theory of cortical micro-circuits," PLoS Computat. Biol.,
vol. 5, no. 10, p. e1000532, 2009.
IEEE ComputatIonal IntEllIgEnCE magazInE | may 2016
[26] W. Maass and C. M. Bishop, Pulsed Neural Networks.
Cambridge, MA: MIT, 1998.
[27] M. S. Jensen, R. Azouz, and Y. Yaari, "Spike afterdepolarization and burst generation in adult rat hippocampal CA1 pyramidal cells," J. Physiol., vol. 492, pp.
199-210, 1996.
[28] J. O'Keefe and M. L. Recce, "Phase relationship
between hippocampal place units and the EEG theta
rhythm," Hippocampus, vol. 3, no. 3, pp. 317-330, 1993.
[29] B. C. Lega, J. Jacobs, and M. Kahana, "Human hippocampal theta oscillations and the formation of episodic
memories," Hippocampus, vol. 22, pp. 748-761, 2012.
[30] N. Axmacher, F. Mormann, G. Fernández, C. E.
Elger, and J. Fell, "Memory formation by neuronal synchronization," Brain Res. Rev., vol. 52, no. 1, pp. 170-
182, 2006.
[31] J. Kami´n ski, A. Brzezicka, and A. Wróbel, "Shortterm memory capacity (7+/-2) predicted by theta to gamma cycle length ratio," Neurobiol. Learn. Memory, vol. 95,
no. 1, pp. 19-23, 2011.
[32] Q. Yu, H. Tang, K. C. Tan, and H. Li, "Rapid feedforward computation by temporal encoding and learning with spiking neurons," IEEE Trans. Neural Networks
Learning Syst., vol. 24, no. 10, pp. 1539-1552, 2013.
[33] G. Q. Bi and M. M. Poo, "Synaptic modification
by correlated activity: Hebb's postulate revisited," Annu.
Rev. Neurosci., vol. 24, pp. 139-166, 2001.
[34] R. C. Malenka and M. F. Bear, "LTP and LTD:
An embarrassment of riches," Neuron, vol. 44, no. 1, pp.
5-21, 2004.
[35] J. W. Newcomer and J. H. Krystal, "NMDA receptor
regulation of memory and behavior in humans," Hippocampus, vol. 11, no. 5, pp. 529-542, 2001.
[36] S. Schreiber, J. Fellous, D. Whitmer, P. Tiesinga,
and T. Sejnowski, "A new correlation-based measure of
spike timing reliability," Neurocomputing, vol. 52-54, pp.
925-931, 2003.
[37] S. Fusi and L. F. Abbott, "Limits on the memory
storage capacity of bounded synapses," Nature Neurosci.,
vol. 10, no. 4, pp. 485-493, 2007.
[38] J. Chrol-Cannon and Y. Jin, "Computational modeling of neural plasticity for self-organization of neural
networks," BioSystems, vol. 125, pp. 43-54, 2014.
[39] J. Chrol-Cannon and Y. Jin, "Learning structure of
sensory inputs with synaptic plasticity leads to interference," Front. Computat. Neurosci., vol. 9, no. 103, 2015.
[40] M. E. Hasselmo, C. Bodelón, and B. P. Wyble, "A
proposed function for hippocampal theta rhythm: Separate phases of encoding and retrieval enhance reversal
of prior learning," Neural Computat., vol. 14, no. 4, pp.
793-817, 2002.
[41] S. Kunec, M. E. Hasselmo, and N. Kopell, "Encoding and retrieval in the CA3 region of the hippocampus:
A model of theta-phase separation," J. Neurophysiol.,
vol. 94, no. 1, pp. 70-82, 2005.
[42] O. Jensen, M. Idiart, and J. E. Lisman, "Physiologically realistic formation of autoassociative memory in
networks with theta/gamma oscillations: Role of fast
NMDA channels." Learn. Memory, vol. 3, no. 2-3, pp.
243-256, 1996.
[43] D. Wang and M. Arbib, "Complex temporal sequence learning based on short-term memory," Proc.
IEEE, vol. 78, no. 9, pp. 1536-1543, 1990.
[44] J. Starzyk and H. He, "Anticipation-based temporal sequences learning in hierarchical structure," IEEE
Trans. Neural Networks, vol. 18, no. 2, pp. 344-358, 2007.
[45] B.-T. Zhang, "Hypernetworks: A molecular evolutionary architecture for cognitive learning and memory,"
IEEE Computat. Intell. Mag., vol. 3, no. 3, pp. 49-63, 2008.
[46] E. Neftci, J. Binas, U. Rutishauser, E. Chicca,
G. Indiveri, and R. Douglas, "Synthesizing cognition in
neuromorphic electronic systems," Proc. Natl. Acad. Sci.
USA, vol. 110, no. 37, pp. E3468-E3476, 2013.
[47] Q. Ren, J. Xu, L. Fan, and X. Niu, "A gim-based biomimetic learning approach for motion generation of a multi-joint
robotic fish," J. Bionic Eng., vol. 10, no. 4, pp. 423-433, 2013.
[48] B. Zhao, R. Ding, S. Chen, B. Linares-Barranco,
and H. Tang, "Feedforward categorization on AER motion events using cortex-like features in a spiking neural
network," IEEE Trans. Neural Networks Learning Syst.,
vol. 26, no. 9, pp. 1963-1978, 2015.
Table of Contents for the Digital Edition of Computational Intelligence - May 2016
Computational Intelligence - May 2016 - Cover1
Computational Intelligence - May 2016 - Cover2
Computational Intelligence - May 2016 - 1
Computational Intelligence - May 2016 - 2
Computational Intelligence - May 2016 - 3
Computational Intelligence - May 2016 - 4
Computational Intelligence - May 2016 - 5
Computational Intelligence - May 2016 - 6
Computational Intelligence - May 2016 - 7
Computational Intelligence - May 2016 - 8
Computational Intelligence - May 2016 - 9
Computational Intelligence - May 2016 - 10
Computational Intelligence - May 2016 - 11
Computational Intelligence - May 2016 - 12
Computational Intelligence - May 2016 - 13
Computational Intelligence - May 2016 - 14
Computational Intelligence - May 2016 - 15
Computational Intelligence - May 2016 - 16
Computational Intelligence - May 2016 - 17
Computational Intelligence - May 2016 - 18
Computational Intelligence - May 2016 - 19
Computational Intelligence - May 2016 - 20
Computational Intelligence - May 2016 - 21
Computational Intelligence - May 2016 - 22
Computational Intelligence - May 2016 - 23
Computational Intelligence - May 2016 - 24
Computational Intelligence - May 2016 - 25
Computational Intelligence - May 2016 - 26
Computational Intelligence - May 2016 - 27
Computational Intelligence - May 2016 - 28
Computational Intelligence - May 2016 - 29
Computational Intelligence - May 2016 - 30
Computational Intelligence - May 2016 - 31
Computational Intelligence - May 2016 - 32
Computational Intelligence - May 2016 - 33
Computational Intelligence - May 2016 - 34
Computational Intelligence - May 2016 - 35
Computational Intelligence - May 2016 - 36
Computational Intelligence - May 2016 - 37
Computational Intelligence - May 2016 - 38
Computational Intelligence - May 2016 - 39
Computational Intelligence - May 2016 - 40
Computational Intelligence - May 2016 - 41
Computational Intelligence - May 2016 - 42
Computational Intelligence - May 2016 - 43
Computational Intelligence - May 2016 - 44
Computational Intelligence - May 2016 - 45
Computational Intelligence - May 2016 - 46
Computational Intelligence - May 2016 - 47
Computational Intelligence - May 2016 - 48
Computational Intelligence - May 2016 - 49
Computational Intelligence - May 2016 - 50
Computational Intelligence - May 2016 - 51
Computational Intelligence - May 2016 - 52
Computational Intelligence - May 2016 - 53
Computational Intelligence - May 2016 - 54
Computational Intelligence - May 2016 - 55
Computational Intelligence - May 2016 - 56
Computational Intelligence - May 2016 - 57
Computational Intelligence - May 2016 - 58
Computational Intelligence - May 2016 - 59
Computational Intelligence - May 2016 - 60
Computational Intelligence - May 2016 - 61
Computational Intelligence - May 2016 - 62
Computational Intelligence - May 2016 - 63
Computational Intelligence - May 2016 - 64
Computational Intelligence - May 2016 - 65
Computational Intelligence - May 2016 - 66
Computational Intelligence - May 2016 - 67
Computational Intelligence - May 2016 - 68
Computational Intelligence - May 2016 - 69
Computational Intelligence - May 2016 - 70
Computational Intelligence - May 2016 - 71
Computational Intelligence - May 2016 - 72
Computational Intelligence - May 2016 - Cover3
Computational Intelligence - May 2016 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202311
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202308
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202305
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202302
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202211
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202208
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202205
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202202
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202111
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202108
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202105
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202102
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202011
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202008
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202005
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202002
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201911
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201908
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201905
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201902
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201811
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201808
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201805
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201802
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter12
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall12
https://www.nxtbookmedia.com