IEEE Signal Processing - May 2018 - 104

system with a predefined transfer function and codebooks
whose parameters are estimated from the input signal, whereas
CSC is an NN system with learned parameters using a machine-learning approach. Concerning temporal context, LPC
has a synchronous (fixed) subframe structure, a wider frame of
20 ms for calculation of the p LPC coefficients, and a narrow
frame of 5 ms for estimation of the LPC excitation signal u. On
the contrary, speech communication is known to be an asynchronous process due to the asynchronous evolution of various
articulatory feature streams [16].

Target speech representation
Speech recognition and synthesis are currently highly dependent
upon machine-learning tools and big data, and speech coding
stands to benefit from it (e.g., by learning better speech representation) [76]. However, there is almost no utilization of the overall
code (underlying structure) of spoken language. The notion of a
code implies relations between message units and signal units
[77]. Articulatory interpretation of auditory spectrograms is a
key to its understanding; however, it becomes more elusive
when applied to brain function. It can be hypothesized that, once
the speech code is deciphered, very effective speech compression algorithms can be designed, approaching the efficiency of
cognitive processing that operates at about 50 bits/s [1].

CB1
< 100 Hz

CB2
100-200 Hz
.
.
.

CB24
12-15.5 kHz
CB25
> 15.5 kHz
(a)
PF1
Anterior

PF2
Nasal
.
.
.

PF12
Strident

NN-based speech coding
Deng et al. in [85] have demonstrated that deep autoencoders
can discover some good, discrete representations or codes for
the entire speech spectrum. The proposed autoencoder was
designed as a deep, five-layer network, with a middle coding
layer where the real-valued activations of hidden units are
quantized to be either zero or one, with 0.5 as the threshold.
These binary codes are then used to reconstruct the original
spectrogram. The authors showed improvements over a conventional vector quantization coder with the Linde-Buzo-
Gray algorithm. The binary nature of the code resembles the
binary nature of phonological speech representation, which is
believed to be crucial in the organization of the speech sounds
in human brains [25].
Phonological features lie on low-dimensional subspaces.
These low dimensions pertain to either physiological structures of the speech production mechanism or the linguistic
structures of the suprasegmental information. At the physiological level, only certain (very few) combinations of the phonological features can be realized through human vocalization.
This property can be formalized by constructing a codebook
of structured sparse codes for phonological feature representation. Likewise, at the linguistic level, only some (very few)
suprasegmental (e.g., syllabic) mapping of the sequence of phonological features is linguistically permissible. This property
can be exploited for block-wise coding of these features with a
slower (suprasegmental) dynamic.

Short-term physiological coding

PF13
Round
(b)

Figure 6. A representation of channel speech coding. (a) Channels
in subband coding [64] are composed of the critical band filters,
inspired by Homer Dudley's channel vocoding [86] from 1939.
(b) Channels in subphonetic coding [55] are composed of the
NN-based phonological filters DNNs. Phonological features are
also known as distinctive or phone-attribute features.
104

The speech code representation is usually studied in neurolinguistics [78], [79] without the necessary technology transfer to communication engineering. Both the motor theory [80]
and direct realist theory [79] of speech perception claim that
the same set of invariants is shared in speech perception and
production. The existence of invariant speech representation
is greatly debated in motor control, psycholinguistics, neuropsychology, and speech neuroscience. Recent findings suggest
that this representation is based on auditory and somatosensory
speech production parameters [81] and known time-varying
articulatory gestures [26], [33], [82], [83].
There is currently no analytic solution for the speech code
representation, however, neuropsychological and brain-imaging work indicates that language learning produces dedicated
NNs that code the patterns of native-language speech [84].
This also has led speech engineers to investigate NN-based
speech coding.

Figure 6 shows a well-known channel speech coding
scheme. Subband coding splits the signal into different frequency bands, imitating the human auditory system. Similarly, sub-phonetic coding splits the signal into different
phonological classes, imitating phonological processing of
the central auditory system. Each phonological class leaves an
acoustic signature that listeners can track, as similarly shown
in [79]. Parallel feature transmission facilitates asynchronous
streams evolution.

IEEE Signal Processing Magazine

May 2018

Table of Contents for the Digital Edition of IEEE Signal Processing - May 2018