Signal Processing - March 2017 - 91

68%

ER
VB

67%

VB
C

-B

D
M

R
D

-B

63%

57%

56%

M
R

50%

Here, results from a pilot user study with 20 participants are
presented. While none of the participants has used an intentaware display before, the study employs a prototype in-car intelligent predictive touch-screen system that performs intent
inference in real time and seamlessly implements the midair
selection facilitation scheme as discussed in the section "Facilitation Scheme and Decision"; see the supplementary material in
IEEE Xplore for a demonstration. An audible cue, i.e., a short
ping sound signal, is produced by the predictive display to con-

40
30
20 52%
10
0

Real-time results from a prototype system

Average Aggregate
Correct Predictions (%)

Mean Destination
Successful Prediction (%)

firm to the user that an interface icon has
refine the applied model parameters from
The reviewed concept
been autoselected. The subjective workload
the larger available data set(s). This can
of intelligent predictive
of interacting with an in-vehicle touch screen
result in a more accurate modeling and predisplays in this article
with and without the predictive functionality
diction procedure.
presents itself as a
is recorded using the NASA TLX test [31],
In Figure 6, the linear destination repromising smart HMi
which is widely utilized in HMI-HCI studverting, Brownian motion, and constant
ies. It requires the participant to complete a
velocity models with the bridging prior,
technology.
questionnaire to rate and weight the mental,
notated by MRD-BD, ERV-BD, BM-BD,
physical, and temporal demand as well as performance, effort,
and CV-BD, are assessed. A mean reverting diffusion
and frustration experienced when carrying out the in-vehicle
model without bridging, MRD, is also examined. Figure 6
pointing tasks. The durations T of accomplishing selection
also depicts the outcome of the probabilistic nearest neighbor (NN), which assigns the highest probability to a GUI
tasks in the trials are also assessed. This can be viewed as an
item closest to the current position of the pointing fingertip as
objective measure of the effort involved.
per p ^Yn D I = D ih = N ^Yn; d i, C NNh, and bearing angle (BA)
When the predictor is off, the trial is a classical experiment
of interacting with a conventional touch screen, where the user
where p ^Yn Yn - 1, D I = D ih = N ^i i, n; 0, v 2BAh [10]. The latter
has to touch the display surface to select a GUI icon. With the
assumes a minimal cumulative angle to the destination located
prediction and midair selection functionality on, the intentin 3-D at di; C NN is the covariance of the multivariate Gaussian
aware predictive touch screen often executes the selection
distribution and i i, n _ + ^Yn - Yn - 1, d ih is the angle to D i ! D.
action for the user. An autoselection action is triggered at time
Figure 6 illustrates that the BD-based inference models,
CV-BD and ERV-BD, achieve the earliest successful predict k G T once the calculated probability for a given GUI icon, as
tions, since they capture the importance of the velocity compoper the estimated p (D I = D i Y1: k), exceeds a set threshold
nent. This is particularly visible in the first 70% of the pointing
for a predefined period of time (on average, threshold c = 0.55
task in Figure 6(a), where a pointing facilitation scheme can
be most effective. Destination prediction toward the end of the
pointing gesture can have limited impact, since by that stage the
100
user would have already dedicated the necessary attention/effort
90
to execute the selection task. The performance of all depicted
80
predictors generally improves as the pointing finger is closer to
70
the display. This is particulary visible for the NN model, which
60
is built on the premise that the pointing finger is closest to the
NN
50
intended end point. An exception is the BA model, since the
BA
reliability of i n as a intent measure declines as t n " T . Over40
MRD
all, this figure shows that probabilistic predictors can success30
BM-BD
MRD-BD
fully infer the intended destination on the display remarkably
20
CV-BD
early in the freehand pointing gesture. For example, in 60% of
10
ERV-BD
cases, the bridged ERV model, ERV-BD, can infer the true intent
0
20 30
40
50
60
70 80
90 100
only 40% into the pointing gesture (with overall correct deciPercentage of Pointing Gesture Time (100*tk /T )
sion exceeding 65%) thus, it can reduce pointing time-effort
(a)
by over 60%.
The gains of combining the MRD motion model with the
100
bridging method are noticeable in Figure 6(a). This is due to to
90
the ability of bridging technique (the prior on X T ) to reduce the
80
sensitivity of LDR models to variability in the processed tracks;
70
60
it tapers the system sensitivity to parameter estimates and the
50
parameter training requirements.

(b)

Figure 6. The end-point inference performance with a MAP estimate as a
function of the percentage of pointing time [28]. (a) The mean successful destination inference. (b) The gesture portion (in time) with successful predictions.

IEEE SIgnal ProcESSIng MagazInE

March 2017

Table of Contents for the Digital Edition of Signal Processing - March 2017