IEEE Signal Processing - May 2018 - 106

1) CSC model architecture: What type, size, and composition of NNs are suitable for CSC?
More than 77% of all coding distortion of NN codec comes
from the parametric vocoding used for speech resynthesis.
This challenge can, therefore, be inspired by recent advancements in end-to-end speech synthesis research. Modeling of
the long-term context in CSC can be investigated by adapting spiking or recurrent NNs.
2) CSC speech parameters: What is an efficient speech representation for CSC, and how is its sparse coding designed?
CSC speech parameters should be based on cortical representation introduced in the "Human Cognitive Speech
Processing" section. It includes sparse short-term (physiological) and long-term (linguistic) representations. The parameters
have to be efficiently estimated by the CSC model.
3)
CSC adaptability: How is computational sensory feedCurrent challenges in cognitive speech coding
back realized?
The following outlines current challenges in cognitive
The goal is to implement an adaptation of the cortical
speech coding.
speech representation based on sensory feedback. In general, it consists
of articulatory gestural feedback
Short-Term Analysis
Training
Classification
and auditory feedback. The latter is
broadly related to auditory targets
Z
used in the nonintrusive estimation
MFCC
of speech quality, such as the ITU-T
recommendation P.563 that defines
a perceptual model of speech. CSC
Backpropagation
Forwardintroduces a novel feedback for
Propagation
the speech code constructed from
Force-Alignment Labels
the auditory and articulatory targets,
(a)
currently not used by waveform/
LPC coding.
Training
Long-Term (Pitch) Analysis
Syllabification
An example of adaptive speech
E
activity
detection has been demonE
Energy
Pitch
strated
[44].
The system employs a
+
Parameters
2-D
Gabor
filter
bank whose paramMFCCs
I
I
eters are retuned offline to improve
the separability between the feature
Spike
InhibitoryTrains Update
representation of speech and nonSpike Bursts
Syllable Boundaries
speech sounds, and it attempts to
(b)
minimize the misclassification risk
of mismatched data, with respect to
Short-Term Analysis
Training
Regression
the original statistical models.
4) CSC robustness: How is attention
...
focused on CSC to increase intelligiSynthesis
bility or to decrease cognitive load?
CSC benefits from the biological cogAcoustic Targets
nitive aspects of speech communiBackpropagation
Forwardcation by trying to design speech
Propagation
Acoustic
codecs that are aware of the hidden
Parameters
structure of the transmitted speech sig(c)
nal (speech code), while the transmitted code remains linguistically
Figure 8. (a) The training and inference stages for an analysis DNN, (b) a spiking NN, and (c) a synrelevant. Linguistically relevant transthesis DNN. Training of the analysis DNN and the spiking NN requires phonetic and syllabic boundary
mission codes bring novel functionallabels, respectively, whereas training of the synthesis DNN does not require force-aligned labels. Pitch
ity to speech transmission systems,
is the fundamental frequency of the sound signal [90].
106

Nasal

Decoding is realized as a synthesis DNN that learns the
highly complex mapping of the transmitted sequence, Z, to
the speech parameters, and consists of two computational
steps. The first step is a DNN forward pass that generates
the speech parameters, and the second step is the generation of the speech samples from the speech parameters
including a decoded pitch signal. The pitch signal is coded
with a codebook that contains the logarithm of the continuous pitch of all of the syllables of the training data, parametrized with the discrete Legendre orthogonal polynomials.
Intelligibility evaluation of this NN codec demonstrates
about 10% degradation, when comparing with LPC (Speex)
coding; however, it operates at a bit rate approximately six
times lower.

IEEE Signal Processing Magazine

May 2018

Table of Contents for the Digital Edition of IEEE Signal Processing - May 2018