Signal Processing - November 2016 - 55

One basic challenge is determining the addressee of an
open 1-2 inches traveling at 65 miles/hour can be very close
utterance, since a driver (or passenger) may be speaking to
to speech level in the frequency band 0-1 kHz. Noise is not
the system, or to another person, or to a person outside the
only intrusive and reduces the drivers' concentration but also
car (e.g., on a call), or even to an automatic assistant on,
degrades the ASR performance and interrupts the dialog flow.
e.g., his or her mobile phone. A system needs to know when
While some noise conditions are quite similar across vehicle
it is being addressed, and respond only
types, some in-vehicle noise conditions
then and not in other contexts. In addivary significantly across vehicles such as
Another challenge for
tion, the interruption of conversation will
noise produced by engines, turn signals,
in-vehicle speech
generally need to put another "on hold"-
or wiper blades. Within the same vehicle,
something that people are used to doing
noise levels can vary from very low when
recognition and
with other people, but generally not with
the engine is idle and windows are closed,
understanding
systems. Addressee detection will need
to very high when traveling at high speeds
comes from imperfect
to scale to be able to suspend as well as
with open windows. Similar to other dialog
speech input.
resume conversations. It is worth noting
systems, environmental noise also impacts
that the current practice of "hot" words
how speakers speak, affecting a wide range
(wake-up words that are used to engage with a system)
of speech characteristics (including task stress, emotion, and
serves to initiate an addressee, but not to effectively susLombard effects).
pend, resume, or close these interactions.
Another challenge for in-vehicle speech recognition and
Even given the correct addressee, another challenge is
understanding comes from imperfect speech input. When
how to handle interruptions to fluency of incoming speech
drivers are under stress, their speech can be less fluent and
for the timely determination of system responses. As just
predictable. Their speech tends to contain more word fragnoted, speech contains pausing and disfluencies [34], and
ments, restarts and repairs, hesitations, and alternative phrasthe task of driving only enhances opportunities for distracings. These deviations from standard speech result in degraded
tion and coping with sudden changes in the car or surroundspeech recognition performance as a result of variation in
ing environment. Such fluency breaks cause important
acoustics and language.
ambiguities for turn-taking. For example, even a simple
For acoustic modeling, the classical cross-word modelpause to a navigation system, as in "which road do I turn
ing approach becomes less effective due to word fragmenonto (long pause) after I cross the bridge" produces differtation and hesitation. Similarly, speech endpoint detection
ent results pre- (incorrect) versus post- (correct) pause that
becomes very difficult as the engine does not know whethmatter to timely interaction. Waiting induces system latency
er a silence is a long pause or the end of a request, espeif the speaker was actually done. Work using acoustic-procially in the presence of noise. For language modeling,
sodic features of prepause speech [35], as well as incremenword fragments pose a challenge since there are often many
tal content [36], can be used to better determine whether a
proper names to cover and names can be interrupted leavuser is suspending versus finishing an utterance. Additional
ing only a fragment. The more word fragments are includchallenges exist for handling self-repairs [37] in the driver's
ed in the system, the more confusability is added, and it is
speech in real time.
harder to build a low-perplexity language model to conFuture challenges in conversational management also
strain search space.
include the system production of "conversational grounding,"
Initial attempts have been made to improve the detection of
which becomes more necessary as utterance length and comdisfluent speech [32]. As DNN-HMM and Connectionist Templexity increases. In natural conversation, partners employ
poral Classification (CTC) are overtaking GMM-HMM [18],
speech back channels such as "uh-huh," as well as visual cues
[33], DNN-based technologies are predicted to better handle
such as gaze and head nods, to convey to each other that they
disfluent speech although their potential benefits need to be
are "still listening." Mobile interfaces currently display visual
validated through real-world in-vehicle uses.
information to ground users, but as autonomy increases, audio
rather than visual grounding offers the benefit of an eyes-free
Challenges in coordinating conversations
option. Research on system-produced back channeling [38]
Beyond the challenges of word recognition itself, interactoffers promise for the future of natural interactive systems, but
ing with increasingly capable voice technologies in the car
scaling to grounding in the safety and multiconversation enviwill require more sophisticated coordination of human-
ronment requires a better understanding of how users interact
machine conversations. Drivers freed from lower-level
with system-produced grounding mechanisms in real time and
driving tasks will have more opportunity for social interacunder cognitive load.
tions both inside and (via mobile) outside the car. And the
For all of these conversational management tasks, the
systems in the car that they do communicate with will have
in-vehicle environment offers unusually rich opportunities
advanced knowledge and complexity. As a result, conversafor speaker-dependent modeling. With fewer lower-level
tion management presents challenges and new opportunidriving tasks to attend to, drivers are expected in particular
ties even if one assumes that high-quality word recognition
to vary dramatically in use of voice for reasons unrelated
is available.
IEEE SIgnal ProcESSIng MagazInE

November 2016

Table of Contents for the Digital Edition of Signal Processing - November 2016