Signal Processing - January 2017 - 70

is to provide assistance to automate
tasks or further the user's interests for
Reactive Assistance
Proactive Assistance
things he or she cares about, all within
context, without explicit user request [8].
"Book_ Taxi"
"Restaurant Suggestions"
User Experience
To achieve that, the agent is designed to
possess a set of attributes; it should be
valuable in that it advances the user's
Data
interests and tasks, while not interfering
Reactive Assistance
Proactive Assistance
Back-End Databases,
with the user's own activities or attenASR, LU, Dialog,
Inferences, User
Services, and Client
LG, TTS
Modeling, Suggestions
tion unless it has the user's explicit
Signals
approval. It should be unimposing. The
agent should be transparent in what it
knows about the user. It should be anticDevice/Service End Points
ipatory and know the future needs of the
(Phone, PC, Xbox, Web Browser, Messaging Applications)
user and bring opportunities to the surface. The agent should also continuously
(a)
(b)
learn and refine its decisions from the
Figure 2. The personal digital agent architecture for (a) reactive assistance and (b) anticipatory computing. feedback signals it receives regarding
the actions it takes. These principles put
the user at the center, and the agent's
actions
are
considered
valuable
only if they ultimately add value
architecture depicts proactive and reactive user experiences,
for the user. Proactive assistance operates on the proactivity
data, and service end points. Reactive assistance is shown in
continuum [31], which ranges from zero to full automation,
Figure 2(a), where the user issues an explicit natural language
allowing for the following scenarios:
command (e.g., "book me a taxi") to the agent. The user request
is handled through a set of reactive assistance components,
■ do it yourself (no help from the agent)
such as speech recognition, LU, and DM. The data coming
■ user tells the agent what to pay attention to (notifications/
from various back ends, and applications are served to the user
alerts)
according to the constraints specified in the natural language
■ agent infers user's habits/patterns and makes suggestions
query. The experience (reactive and/or proactive) can be served
(inference/suggestions)
in one or more of the different device or service end points.
■ agent makes decisions and takes actions (full autonomy on
Proactive assistance [Figure 2(b)] involves anticipatory
task decisions/executions).
computing, where the personal digital agent does things in
Most of the currently supported proactive scenarios are
a contextual manner (i.e., at the right time and place) that it
notifications/alerts and suggestions. Even though there is some
expects is valuable to the user without an explicit user request.
preliminary work, none of the agents in production supports
Proactive assistance makes use of inference, user modeling,
autonomous decision making and action taking on behalf of
and ranking to power experiences. Backthe user without confirmation.
end data, device, applications, and web
The proactive agent system architecture
even though proactive
services signals are leveraged for proactive
is shown in Figure 3. Signals coming from
and reactive parts of the
inference and triggering.
web services, device sensors, and the user's
current PDA architectures
Even though proactive and reactive parts
profile are processed, where processing
of the current PDA architectures are built in
includes parsing, enriching, and filtering
are built in isolation, in
isolation, in principle they can use a single
to merge device and service data. The next
principle they can use
architecture to enable both types of experistep is aggregation, which joins the proa single architecture
ences. In fact, most proactive scenarios have
cessed data streams through time and space
to enable both types of
reactive extensions and vice versa. For exam(i.e., location) about the user's whereabouts
experiences.
ple, if the user makes a restaurant reservation
and actions/tasks done at specific times and
(reactively), the agent may (proactively) sugplaces. This step blends the physical and
gest a movie after the dinner or may offer to book a cab to
digital worlds and allows for powerful inferences that capture
take the user to the restaurant. Data and context are shared
repetitive behavior and events in both worlds. The signals are
between the two assistance modes. Next, we focus on the proused to make inferences and train machine-learned models
active system architecture and the components that power
for modeling the user and his or her interests. The same set
proactive scenarios.
of signals is also used to set rules for notifications and alerts
the user wants the agent to serve. The models and rule recipes are deployed to a run time environment. Once proactive
Proactive assistance
scenarios are deployed in production, capturing and feeding
Proactive assistance is based on the theory of proactivity that
back user behavior signals regarding notifications, alerts, and
describes user desires and a model of helpfulness [7]. The goal
70

IEEE Signal Processing Magazine

January 2017

Table of Contents for the Digital Edition of Signal Processing - January 2017