IEEE Computational Intelligence Magazine - May 2018 - 13

Publication
Spotlight

Haibo He, Jon Garibaldi, Kay Chen Tan,
Graham Kendall, Yaochu Jin, and
Yew Soon Ong

CIS Publication Spotlight

IEEE Transactions on Neural
Networks and Learning Systems

LSTM: A Search Space Odyssey, by K.
Greff, R. K. Srivastava, J. Koutník, B.
R. Steunebrink, and J. Schmidhuber,
IEEE Transactions on Neural Networks
and Learning Systems, Vol. 28, No. 10,
October 2017, pp. 2222-2232.
Digital Object Identifier: 10.1109/
TNNLS.2016.2582924
"Several variants of the long shortterm memory (LSTM) architecture for
recurrent neural networks have been
proposed since its inception in 1995. In
recent years, these networks have
become the state-of-the-art models for a
variety of machine learning problems.
This has led to a renewed interest in
understanding the role and utility of
various computational components of
typical LSTM variants. In this paper, we
present the first large-scale analysis of
eight LSTM variants on three representative tasks: speech recognition, handwriting recognition, and polyphonic
music modeling. The hyperparameters
of all LSTM variants for each task were
optimized separately using random
search, and their importance was
assessed using the powerful functional
ANalysis Of VAriance framework. In
total, we summarize the results of 5400
experimental runs (≈15 years of CPU
time), which makes our study the largest
of its kind on LSTM networks. Our

Digital Object Identifier 10.1109/MCI.2018.2806987
Date of publication: 10 April 2018

results show that none of the variants
can improve upon the standard LSTM
architecture significantly, and demonstrate the forget gate and the output
activation function to be its most critical
components. We further observe that the
studied hyperparameters are virtually
independent and derive guidelines for
their efficient adjustment."
Broad Learning System: An Effective and
Efficient Incremental Learning System
Without the Need for Deep Architecture,
by C. L. P. Chen and Z. Liu, IEEE
Transactions on Neural Networks and
Learning Systems, Vol. 29, No. 1, January 2018, pp. 10-24.
Digital Object Identifier: 10.1109/
TNNLS.2017.2716952
"Broad Learning System (BLS) that
aims to offer an alternative way of learning in deep structure is proposed in this
paper. Deep structure and learning suffer
from a time-consuming training process
because of a large number of connecting
parameters in filters and layers. Moreover, it encounters a complete retraining
process if the structure is not sufficient
to model the system. The BLS is established in the form of a flat network,
where the original inputs are transferred
and placed as "mapped features" in feature nodes and the structure is expanded
in wide sense in the "enhancement
nodes." The incremental learning algorithms are developed for fast remodeling
in broad expansion without a retraining
process if the network deems to be
expanded. Two incremental learning
algorithms are given for both the incre-

ment of the feature nodes (or filters in
deep structure) and the increment of the
enhancement nodes. The designed
model and algorithms are very versatile
for selecting a model rapidly. In addition,
another incremental learning is developed for a system that has been modeled encounters a new incoming input.
Specifically, the system can be remodeled in an incremental way without the
entire retraining from the beginning.
Satisfactory result for model reduction
using singular value decomposition is
conducted to simplify the final structure.
Compared with existing deep neural
networks, experimental results on the
Modified National Institute of Standards
and Technology database and NYU
NORB object recognition dataset
benchmark data demonstrate the effectiveness of the proposed BLS."
IEEE Transactions
on Fuzzy Systems

Improving Supervised Learning Classification Methods Using Multigranular
Linguistic Modeling and Fuzzy Entropy,
by J. A. Morente-Molinera, J. Mezei,
C. Carlsson, and E. Herrera-Viedma,
IEEE Transactions on Fuzzy Systems,
Vol. 25, No. 5, October 2017, pp.
1078-1089.
Digital Object Identifier: 10.1109/
TFUZZ.2016.2594275
"Obtaining good classification results
using supervised learning methods is
critical if we want to obtain a high level
of precision in the classification processes.The training data used for the learning

May 2018 | IEEE CoMputatIonal IntEllIgEnCE MagazInE

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - May 2018