Signal Processing - November 2016 - 54

to engage the driver. Available services (e.g., gas station,
eyes-busy scenarios. Because of the special nature of in-vehicle
or parking) via TTS from users' mobile devices add
use, however, additional challenges are apparent from in-vehicle
another dimension of complexity and are competing for
data collection efforts and in-vehicle dialog system development
drivers' attention. Managing multiple assistance systems
activities mentioned in the section "Review of Past Major Activfrom different devices will become an important developities." Some important challenges are summarized in Table 1.
ment consideration.
One may look at these challenges as arising from the three
interdependent factors: 1) the driver, 2) the environment, and 3)
■ The automotive industry. Two key issues are critical to
the automotive industry.
automotive companies' decisions about whether or not to
adopt new technologies. One is safety. Improper imple■ The driver. Drivers typically have short attention spans durmentation of certain technologies could lead to fatal acciing interaction with in-vehicle dialog systems, as driving is
dents. The other is the reliability. When a
their primary task. From 2020 to 2030,
component is installed in a vehicle, it must
drivers will begin to be exposed to autonThe automotive industry
work properly for a long time. Vehicle
omous driving technologies, freeing them
has taken two different
parts and features installed are typically
from constant attention to vehicle control.
approaches to address
required to function properly for at least
However, according to projections by
the arrival of cloud-based, ten years. For the safety and reliability reaMcKinsey [39], the adoption of self-drivvoice-enabled assistance
sons, this industry is highly regulated by
ing vehicles will likely be less than 15%
systems from major
the government. Relevant regulations and
by 2030. HMI while driving will continue
guidelines include ISO 26262 [29], ISO
to be important challenge. As a result,
IT companies.
15504 (Automotive SPICE), and IEC
CID systems have to handle challenges
61508 [30]. Due to such requirements, common practice
on many different levels [9]. Drivers' speech may be fragin the consumer electronics or Internet world of fast prodmented, disfluent, and repetitive. Drivers may need to hear
uct development cycle may not be directly applied in this
shorter or simpler responses; they may expect the system
industry. In the automotive industry, a rigorous process
to provide straightforward recommendations instead of a
covering design specification, development, testing, and
potentially overwhelming number of choices. When sysvalidation is followed to ensure the resulting product qualtems make recommendations, the systems should anticiity meets requirements. For example, to develop a CID
pate driver's contextual needs to the point that the driver
system, one needs to specify the system coverage and perhas the feeling that she or he doesn't need to state the
formance by listing many phrasal and interactive variaobvious, so driver behaviors and personal preferences
tions under different noise conditions by speakers from
need to be taken into consideration to avoid excessive condifferent dialect regions, leading to a huge number of
versational turns. Additionally, in one vehicle, there may
combined testing cases. A complete testing of these cases
be multiple passengers, each requiring a separate preferagainst various requirements becomes more and more chalence profile. A sophisticated dialog system should be able
lenging, especially with increasing coverage needs from
to coordinate requests from different users, know when to
the users.
not interrupt human-human conversations, and come up
The combination of these three factors causes many techniwith an optimized response for the group.
cal
challenges. We next highlight some critical ones that have
■ The environment. The in-car environment is generally
long-term impact in the areas of speech recognition and undermore dynamic and has higher stakes than the contexts in
standing, multiple speaker conversation coordination, the effect
which other dialog systems are deployed. Inside the vehiof driver behaviors and states for safety, as well as the integration
cle, information from sensors reflects vehicle status
of general intelligent assistance systems.
changes, some of which need immediate attention (an
engine breaking down), while others can be handled at a
later time (an oil change warning). Differences in design,
Challenges in speech recognition and understanding
interior materials, and mechanics create different acousFrom a historical perspective, speech recognition in the car
tic environments and background noises inside vehicles.
started with small vocabulary systems primarily for comOutside the vehicle, the physical environment is also
mand and control, along with optimization of either microdiverse and dynamic. A vehicle may be on a highway or a
phone placement or multimicrophone array processing to
city road, accelerating or decelerating, on gravel or
suppress the diverse noise sources present for in-vehicle
asphalt-creating significantly different background
scenarios [31]. More recent efforts have focused on
noises. Traffic conditions can vary significantly: the drivexpanding speech recognition coverage to additional iner may be stuck in stop-and-go traffic, or may be travelvehicle domains.
ing unimpeded at high speed. Harsh weather, such as
The in-vehicle acoustic environment is complex and dynamwind, rain, hail, or thunder, typically requires additional
ic. Factors in this acoustic environment include noise from
attention from the driver and alters the in-vehicle acousair conditioning units, wiper blades, the engine, external
tics. In such cases, the dialog system may need to use diftraffic, the road surface, wind, open windows, and inclement
ferent dialog strategies with respect to taking the initiative
weather. The level of background noise while windows were
54

IEEE SIgnal ProcESSIng MagazInE

November 2016

Table of Contents for the Digital Edition of Signal Processing - November 2016