IEEE Signal Processing - May 2018 - 107

performing tasks such as automatic dialect correction and
pronunciation improvements for people with phonological and
articulatory disorders that might eventually lead to intelligibility
enhancements.
Error minimization can perform error correction as
defined by the DIVA and HSFC models. Consider the task
of dialect correction, for example. Error minimization might
be implemented as an automatic accentedness evaluation of
nonnative speech [91], [92] in a closed-loop fashion.
Cognitive load, related to the cost of cognitive processing resources and listening effort, is currently studied in the
field of cognitive hearing science and by manufacturers of
hearing aid devices. CSC can be inspired by this ongoing
research to apply their results into intelligibility enhancements of transmitted speech signals.
5) CSC language independence: How can NN-based CSC
language be made independent?
Language-independent speech and language technology
belong to the main research topics of the speech signal
processing community. One approach is in modeling all of
the International Phonetic Alphabet (IPA) symbols. There
are more than 100 IPA symbols that can be further extended with more narrow phonetic and prosody annotations.
On the contrary, all of the speech sounds share some attributes, known in linguistics as phonological features or
phone attributes in the speech community. In phonology,
one example could be the sound pattern of the English set
consisting of 13 features [93]. In the speech community,
one major effort resulted in the Automatic Speech
Attribute Transcription project [94], which proposes a
framework for speech understanding based on processing
of cognitive hypotheses created from the acoustic and
auditory cues of the phone attributes. Recent phonetic
DNN training analysis confirmed that the hidden layers
learn an effective representational basis-the phone attributes-for the formation of invariant phonemic categories
[95]. Exploring this phonetic invariance across the languages is, therefore, a promising direction for the multilingual
CSC technologies.
6) CSC speaker recognizability: How can NN-based CSC
speaker be made independent?
Another very relevant problem is speaker recognizability.
Early work on speaker-adaptive phonetic vocoding resulted
in variable bit-rate coding [96]. Promising results have
shown recent advancements in NN-based generative models of raw audio, such as WaveNet from Google Research
[62] and Deep Voice 2 from Baidu Research [97], capable
of learning and imitating hundreds of voices within a single generative model.
7) CSC complexity: How can the computational complexity
of CSC be minimized?
Complexity of compressive sampling is about O (K log (N/K))
for K-sparse (K % N ) N-dimensional speech representation.
Computational complexity of a DNN is about N w, where
w is the number of weights of the DNN. The previously described compositional NN-based codec consisted of

12 analysis DNN, each trained with 2.46 million parameters, and one synthesis DNN trained with 3.31 million
parameters. Decoding with generative models of raw
audio with based on the advanced NN architectures, composed of deep convolutional and recurrent NNs, might have
significantly higher computational demand, not feasible for
current devices.

Summary and conclusions
This article described the impact of cognitive speech processing on speech compression. The basic concepts of
human cognitive speech processing and how CSC has
impacted current audio and speech coding systems have also
been discussed. Properties of CSC were then presented and
compared to LPC.
CSC has great potential to further impact current speech
coding standards. Deep learning and neural basis of speech
and language processing have tremendously advanced recently, and many of these findings can be applied to the field of
speech coding. This article concludes by outlining current
challenges in CSC, relying on incorporating phonological posteriors and temporal models, and employing speech
production models and cognitive feedback for enhanced
speech compression.

Authors
Milos Cernak (milos.cernak@ieee.org) received his B.S.,
M.S., and Ph.D. degrees in electrical engineering from the
Slovak University of Technology, Bratislava, in 1999, 2001,
and 2005, respectively. He is a lead speech-processing engineer
at Logitech, CTO office, in EPFL Innovation Park, Lausanne,
Switzerland. He received awards for his work in 2010 from the
Slovak Academy of Sciences for Best Research Team of the
Year, and in 2012 from the Slovak Ministry of Education for
research and development. His research interests include
speech signal processing, machine learning for phonetics and
phonology, neural basis of speech production and perception,
and pathological speech processing. He is a Senior Member of
the IEEE.
Afsaneh Asaei (asaeiaf@gmail.com) received her B.Sc. and
M.Sc. degrees in electrical and computer engineering from
Amirkabir (Tehran Polytechnique) and Sharif University of
Technology, Tehran, Iran, in 2002 and 2006, respectively, and
her Ph.D. degree in artificial intelligence from the École
Polytechnique Fédérale de Lausanne, Switzerland, in 2013.
Since 2002, she has been a research engineer on several projects
involving applications of machine learning and signal processing
for pattern recognition, voice detection, localization, separation,
enhancement, recognition, authentication, coding, quality
assessment, and evaluation for neurodegenerative pathology
with a focus on robustness and generalization. She is currently
the head of artificial intelligence at the Center for Innovation and
Business Creation at TU München, Germany. Her research
interests include artificial intelligence and health informatics.
Alexandre Hyafil (alexandre.hyafil@gmail.com) received
his B.Sc. and M.Sc. degrees from École Polytechnique,

IEEE Signal Processing Magazine

May 2018

107

Table of Contents for the Digital Edition of IEEE Signal Processing - May 2018