IEEE Signal Processing - May 2018 - 100

Sounds

Cochlea

Auditory Nerve

Filter Banks
Decomposition

High-Pass and
Nonlinear Filter

Parietotemporal
and Premotor Areas
Speech
Production
Dorsal
Stream

IC + Auditory Cortex
Spectrotemporal
Analysis, e.g.,
Gabor Filters

Auditory Speech
Perception
Enhancement
of Frequency
Selectivity

Low-Pass
Filtering

Cochlear Nucleus

Midbrain

Ventral
Stream

Phonological
Network

Speech
Recognition

Peripheral Auditory System

Ventral Temporal Areas

(a)

(b)

Central Nervous System
(c)

Figure 2. The simplified functional model of human speech processing. (a) The first stages of acoustic treatment-the peripheral auditory system.
(b) The dual-stream model of human speech and language processing [8]. Auditory sensations are processed by the primary auditory cortex in two
processing pathways: 1) the dorsal (back or posterior) stream for speech production and 2) the ventral (front or anterior) stream for speech recognition and understanding. (c) Functional decomposition of the primary auditory cortex [25]. The processing consists of spectrotemporal and phonological
encoding. We can formalize phonological coding, e.g., by articulatory phonology [26], which aims to unify low- and high-dimensional descriptions of a
single (speech) system.

The dual-stream model posits two diverging pathways
emerging from the auditory cortex: along the ventral stream,
including ventral regions of the temporal cortex, where sounds
are mapped to meaning using underlying phonological representations; and along the dorsal stream, including parietotemporal as well as premotor areas, where speech sounds are
associated with articulatory patterns, a direct link between
perceptual and motor representations of speech based on the
phonological speech representation. The former stream supports speech recognition and understanding, whereas the latter stream reflects beneficial effects of auditory feedback on
speech production. While the discrete phonological speech
representation plays a fundamental role in both speech perception and production [28], at the level of the temporal lobe, current understanding favors a hybrid continuous/discrete nature
of speech cognition that accommodates the relevance of the
continuous aspects of speech for explaining certain features of
speech (e.g., prosody).

Phonological

Phonetic

Syllabic

Temporal organization of speech perception

Speech remains mostly intelligible when the spectral content is
replaced by noise, and only the envelope-modulation of signal
energy-is preserved, especially all modulations below 12 Hz
[29], [30]. Thus, temporal regularities of the speech signal play a
key role in speech processing. Based on psychoacoustics and neuroimaging studies, researchers have proposed that intrinsic neural
oscillations are important contributors to the segmenting of speech
along timescales of distinct granularities [10]. Evidence suggests
that the auditory cortex segregates acoustic information on at least
three discrete timescales processed in the auditory cortical hierarchy: 1) stress d frequency (1-3 Hz), 2) syllabic i frequency
(4-8 Hz), and 3) phonetic low c frequency (25-35 Hz) [31], with
a strong asymmetry between left and right hemispheres [32].
Functional organization of the ventral sensorimotor cortex supports the gestural model developed in articulatory phonology. Analysis of spatial patterns of activity showed a hierarchy of network
states that organizes phonemes by articulatory-bound phonological features [33]. Building upon temporal
information segregation in the auditory
cortex, Figure 3 shows a structural and
temporal organization of a bottom-up
Timescale
Stressed
organization of human speech perception.
Accented

Figure 3. A different-time granularity of speech processing. The phonological and phonetic classes are
segmental attributes, whereas the syllable type, stress, and accent are linguistic events recognized at
suprasegmental levels [52].

100

IEEE Signal Processing Magazine

May 2018

Top-down control
and speech predictions
There is a long-standing debate about
the functional significance of the massive

Table of Contents for the Digital Edition of IEEE Signal Processing - May 2018