IEEE Signal Processing - May 2018 - 102

Audio and speech coding
(Audio)
Perceptual
Coders

Perception
Model
Production
Model

(Speech) Linear
Predictive
Coders

Figure 4. The underlying models of audio and speech machine coding,
similar to human speech coding shown in Figure 2. Two distinct pathways (technologies) exist: perceptual-model-based (audio) coders and
production-model-based (speech) coders.

Normalized
Input Amplitude Values

1
0.5

µ-Law Compression
A-Law Compression

Perception model-based compression

-0.5
-1
-1

-0.5
0
0.5
1
Normalized Compressed Amplitude Values

Figure 5. A comparison of n-law and A-law algorithms. The n-law
algorithm, primarily used in North America and Japan, provides a slightly
larger dynamic range than the A-law, which is primarily used in Europe.
Typically, a 16-bit digital audio signal is reduced to 8 bits by n-law or
A-law encoding.

information at multiple temporal scales directly mapping to
the syllabic and stress information.
Phonological features, known also as distinctive and phoneattribute features, are considered as a lower-dimensional,
structural representation of phonetic features, analogous to a
red, green, blue (RGB)-color model, in which RGB lights are
added together to reproduce colors. Articulatory phonology
aims to unify the low-dimensional (abstract) and high-dimensional (physical) description of a speech system, in which lower-dimensional articulatory gestures are linguistically relevant.
Bouchard et al. [33] also claim that functional organization of
the ventral sensorimotor cortex supports the gestural model
developed in articulatory phonology.
In the context of speech coding, the short-term speech
representation inferred from the speech signal using a
deep-learning approach (a vector of phonological posterior
features), is shown to enable high compressibility [53], considered to be partially related to articulatory gestures, and
thus, correlates to phonological processing performed in the
STG. The hypothesis of the correspondence of the phonological posteriors to the gestural trajectories is also motivated
by the analogy to the constriction dynamics model [54] that
takes gestural scores as input and generates articulator trajectories and acoustic output. Unlike this constriction dynamics
model, acoustic output is generated using a phonological synthesis described in [55].
102

Machine speech coding as analogous to the dual-stream human
speech coding model is shown in Figure 4. The transmitted
code consists of short-term filter bank parameters (in perceptual
coding), or linear predictive parameters (in speech coding),
and long-term (prosodic) parameters. Temporal parameters are not encoded directly and phonological analysis is not
performed at all. Also observed were two processing (technological) pathways, split according to the underlying human
perceptual or production model. There is only a small overlap
in both technological pathways leading to two distinct classes of
speech coding: perceptual for audio sources and linear predictive
coders for speech sources. Currently, there is no need for a joint
or universal coding scheme; rather, both coding classes are
sophistically switching in realtime according to the source type.
This hybrid approach currently offers the best audio and speech
coding available [60].

Early attempts to incorporate human speech perception into
speech compression were in modeling of auditory masking performed by the inner ear and the cochlea. Modeling of dynamic
masking associated with cochlear outer-hair-cell processing
[61] resulted in the development of n-law and A-law speech
compression algorithms, standardized by the International
Telecommunication Union-Telecommunication Standardization
Sector (ITU-T) G.711 standard released in 1972. These dynamic range speech compression algorithms are still popular, e.g.,
Google's WaveNet, a deep generative model of raw audio
waveforms [62], compresses raw audio using the n-law algorithm before further processing by NNs.
Both n-law and A-law algorithms perform nonlinear
dynamic range compressions designed to reduce the number
of bits of information in each sample of a digital audio signal,
while preserving the dynamic range of samples at low amplitudes. Figure 5 shows the compression of normalized amplitudes of the input samples, which is approximately linear at
low amplitudes and highly nonlinear at high amplitudes.
In later studies, perceptual limitations of the human ear were
used to encode arbitrary signals [63] that evolved into perceptual
audio coding in use today, as exemplified by the moving picture
experts group (MPEG) standards. These coders incorporate several psychoacoustic principles, including absolute hearing thresholds, critical band frequency analysis, simultaneous masking, the
spread of masking along the basilar membrane, and temporal
masking [56]. Imitating the human auditory system, subband
coding breaks the signal into a number of different frequency
bands [64]. This coding does indeed resemble the spectrotemporal feature organization of A1 discussed in the "Audio and Speech
Coding" section. Subband adaptive differential pulse-code modulation with a bit rate of 64 kilobits/s is standardized as the G.722
codec. It is also a key component of the popular MP3 format.
Perceptual coding is also termed open loop, since there is no feedback from the output to the input.
In addition, perceptual speech quality assessment is an
established area of speech coding designed to automatically

IEEE Signal Processing Magazine

May 2018

Table of Contents for the Digital Edition of IEEE Signal Processing - May 2018