Signal Processing - September 2017 - 194

proven surprisingly difficult to translate
findings in one emotional database to
another-a formidable hurdle in developing a real-world system. One viable
approach is to use transfer learning,
wherein similar learning tasks are used
to bias models to learn more quickly
how to perform related tasks. Moreover,
analyzing multimodal temporal signals
comes with the signal processing challenge of synchronization and addressing
diversity in timescales.
Another crucial question from the
behavioral science perspective is how
to group behaviors and know when behavior shifts-and thus a person's latent
state (e.g., mood) may have changed.
Given appropriate multimodal, longitudinal data, computational researchers
can devise mathematical formulations
to differentiate normal data variability
and anomalies from medically pertinent
behavioral transitions. Relatedly, it is of
interest to know what types of behaviors
co-occur and whether there are subpopulations within a disorder that exhibit
similar tendencies; novel clustering approaches that deal with disparate data
types can bring new insights.
Finally, the choice or formulation of
a target behavior representation is not
always straightforward. Often we rely
on human annotation, but this is inherently biased and typically adds another
layer of imperfection and variability to
the modeling task. Much care must be
taken in deciding on a reference behavior of interest. Once a construct is chosen, various mathematical approaches
have been proposed to leverage the reality that different raters are more reliable
in unique situations (e.g., [23]).

Building community among
researchers
The most critical step that computational
and behavioral science researchers can
take is to develop intimate, sustained,
trusting, and productive partnerships
from the early stages of a cross-disciplinary research program. Creating
technological solutions is a strenuous
process that can be wrought with failure
points, and these are only multiplied for
interdisciplinary projects. Collaborators
should communicate often and foster
194

intellectual openness. Furthermore, in
this age of interdisciplinary research,
there is an opportunity to construct educational programs that jointly train collaborative researchers in both clinical and
computational domains. At the intersection of such collaboration will emerge a
new team of scientists who have in-depth
knowledge in multiple domains and will
lead the charge in technology transfer.

Our greatest standing challenge
This brings us to the most imperative challenge that we face in BSP: how to go beyond human abilities, while maintaining
human interpretability. Following the standard of practice, we can use BSP to automate behavioral coding (e.g., measurement
of empathy [21]). Such systems can bring
an expert's opinion (possibly a collection
of experts) to a wider audience, automatically, faster, at low cost, and with a certain
objectivity/consistency. In specific cases,
systems can be independently employed,
such as in the example of providing feedback to a therapist on empathic skills. But
in many other cases, the human expert will
still be next-to-the-loop with their own
opinion, which may be equally valid to the
automated one (i.e., when the automated
system matches inter-human agreement).
As we move the field forward, we must
create automated systems that profoundly
augment human capabilities, effectively
integrating into normal workflows.
Fully autonomous perceptual systems would open previously unimagined
translational potentials. But how can we
possibly go beyond human perception if
we don't use human perception for modeling or validation (noting that these systems with independent perceptual outputs
would likely still rely on expert utilization)? This is an open challenge for which
we are only beginning to find problemspecific solutions. As discussed previously,
one approach is to define a computational
construct in a top-down manner, and then
compare it to peripheral constructs or outcome measures. Recall Lee et al.'s knowledge-based measure of vocal entrainment
[18], for which there is no reliable quantitative perceptual measure; validation was
made indirectly through (hypothesized)
correlation with couple relationship quality. Such top-down approaches require a
IEEE SIGNAL PROCESSING MAGAZINE

September 2017

certain faith in the system's design. In fact,
this approach resembles the design of psychometric instruments; thus, similar methods of reliability testing and validation can
be employed. Still, autonomous system
design is the most challenging and critical
problem we should address.

Vision for the future
Computational science undoubtedly has
much to contribute to human behavioral
study, and this is an exciting time to be a
signal processing researcher. It will take
a joint, collaborative effort to solve these
great challenges posed by mental health
research and clinical needs. Primary
computational targets include multimodality and interaction modeling as well
as behavior (change) prediction. If we can
overcome the engineering obstacles, we
can provide enduring scientific advances
and translational impact in mental health
domains. Particularly one special aspect
of signal processing-in the service of improving mental health and performance-
is the curious fact that the brain may itself
be a signal processor. In other words,
many of the insights garnered in machine
learning and signal processing can be applied to the functioning of the brain in and
of itself. Beyond the practical benefits of
data assimilation surveyed above, there
may be deeper theoretical contributions of
signal processing to our understanding of
things like theory of mind.
An achievable dream of ours is to
see engineering technologies integrated
within and supporting all aspects of mental health research and care, helping to
fill scientific knowledge gaps, connecting
dots, and supporting novel interventions.
Signal processing will enable access to a
truly dynamic, patient-centric care. With
cloud-based architectures and reasonable
cost, these technologies can operate on
a global scale, overcoming cultural and
other boundaries and variables.

Authors
Daniel Bone (dbone@usc.edu) received
his B.S. degrees in electrical and computer
engineering from University of Missouri
in 2009. He received his Ph.D. degree in
electrical engineering in 2016 from the
University of Southern California (USC),
where he is a postdoctoral researcher in

Table of Contents for the Digital Edition of Signal Processing - September 2017