Instrumentation & Measurement Magazine 24-6 - 18

authentication must be considered. Secure storage infrastructure
is also an issue, as well as the question of how long the
audio data must be conserved, and at what resolution. A strategy
to protect data against corruption or loss will be crucial for
longitudinal analyses that span many years.
In terms of processing, the details depend on the exact
audio features under consideration, but some common challenges
remain:
Background subtraction: removing irrelevant ambient noise
from the audio stream. Background speech data, such as from
a television or radio, can pose a particular challenge [23].
Sound classification: speech segments must be distinguished
from non-speech segments, for example, using pitch detection.
Speaker identification: the target speaker's voice must be segmented
from those of visitors or other household members.
Automatic speech recognition (ASR): open-vocabulary ASR
of elderly voices in real-world environments has proven to
be a challenge. In research studies, the data is often manually
transcribed, with the expectation that advances in ASR will be
forthcoming [15], [16], [21]. Other approaches avoid ASR altogether
and focus only on acoustic and prosodic cues [23].
Feature extraction: as seen in Fig. 1, a large number of features
may be extracted from the data. The exact feature set may
depend on factors such as efficiency of processing, non-redundancy
of information, and whether the results need to be
computed in real-time.
Output: machine learning and/or time series analysis is used
to combine speech features and output a measure of cognitive
status, as well as monitor changes over time. To date,
binary classifiers have often been employed, but more sophisticated
approaches are needed. The length of the speech
samples may be highly variable and they might have been
obtained at irregular time intervals. Appropriate machine
learning methods are needed to handle such irregular longitudinal
data.
Finally, any concerning events that are detected (e.g., a
sudden or gradual decline in cognitive ability) must be appropriately
communicated to either the user or, preferably, their
healthcare provider. Sabbagh et al. [19] refer to this process as
" integration into the clinical care pathway. " It is of little use,
and in fact may be psychologically harmful, if a decline is detected
and communicated to the user, but there is no follow up
by a trained professional.
Ethical Challenges
Numerous ethical concerns may be raised by smart home
speech recording; the most commonly-raised concern is privacy
[33]. People have a right to personal privacy, particularly
in their own homes, and systems such as those described above
can lead to psychological discomfort, intrusiveness, and the
18
feeling of " always being watched " [34]. There are also questions
of data privacy, such as who can access the data [33] and
who sees the results of the analysis [19]. Some of these concerns
may be alleviated by technological solutions, such as those described
in relation to a privacy-by-design personal assistant for
seniors [35]. Typically, the current approach has been to transmit
raw speech samples and process them on a remote server.
While this helps with safeguarding the stored data and simplifying
the client side of the system, privacy concerns may be
allayed if only the extracted features are transmitted to the remote
server for generating the desired output. The raw speech
data could be discarded after features are extracted locally.
Another concern is that of informed consent. Obviously,
it would be inappropriate to install an audio surveillance
system in someone's house without their knowledge, but the
notion of informed consent becomes trickier when working
with users who may have, or develop, dementia. Particularly
with the most unobtrusive recording architectures, an
individual may forget, over time, to what they have consented
[34]. There is also the issue of visitors to the home:
users have indicated that they prefer such a system to be
hidden from visitors [33], but at the same time, there is an
obligation to disclose to visitors that they may be recorded
[34]. One possible solution to this is using voice detection to
not record, or immediately delete, recordings from non-consented
speakers [31].
Concerns have also been raised that using the voice as a
digital biomarker may have negative social justice implications.
It is well-known that many AI technologies, including
speech recognition, perform significantly worse for speakers
of languages other than English, people who speak English
with an accent, or speakers of African American English [36].
Furthermore, installing and maintaining smart home technology
can be expensive and therefore not accessible to all. It is
important that such technology not perpetuate existing racial
and socioeconomic inequities in healthcare access [36].
Ways Forward
There is great potential for speech-based measurement of cognitive
status. Many researchers have reported high accuracy
in distinguishing between speakers with and without dementia,
in cross-sectional studies performed under lab conditions.
Although fewer in number, some studies have reported promising
results when recording speech in-situ, through video
chat, wearable devices, and virtual assistants. However, many
gaps remain before we can consider this technology fully
validated.
Many of the studies described above employed at least
some manual processing, either via Wizard-of-Oz dialogues
with participants, manual audio segmentation, or manual
transcription. A fully automated, end-to-end architecture
must be developed, deployed, and evaluated. As previously
mentioned, a thorough accounting of the measurement uncertainty
at each stage in the pipeline must be performed, as well
as an analysis of how those uncertainties propagate through
the pipeline [9].
IEEE Instrumentation & Measurement Magazine
September 2021
Instrumentation & Measurement Magazine 24-6

Table of Contents for the Digital Edition of Instrumentation & Measurement Magazine 24-6
