Instrumentation & Measurement Magazine 26-3 - 40

as electroencephalogram which can involve skull caps and
conductive gel on a user's head or respiratory data collected
from a strap wrapped around a user's upper chest. Some signals
may be slow to change as compared to others, such as
respiratory signals compared to heart rate variability, and
therefore limit a closed-looped system's ability to respond
quickly to a user's change in emotion [14]. Although these
signals may provide informative data that can be related to
emotions [11],[14], their data collection techniques may limit
the type of natural user interactions that can be studied [15]
and may preclude the adoption of these sensors in a user's
daily life [13],[16], thus limiting the future capacity of affective
computing to be applied to everyday life. Therefore, ambulatory
measurement from sensors with a high adoption rate by
users is preferable [8],[17]. Collecting physiological signals
from a wristband device addresses many of the data collection
challenges in affective computing research.
Empatica's E4 is a commercially available wearable device
with embedded sensors that was developed by affective computing
researchers [18] and has been used in several studies
to examine affect and physiological signals [11],[19]. The E4 is
one example of how much physiological-sensing equipment
has evolved in the last decade [8],[16] and matches the characteristics
of an appropriate wearable sensor suggested for
use with the ASD population [20]. Wires have been reduced
or eliminated, and sensors covering several fingertips (which
limited the types of activities a user could do and therefore an
experimenter could study) have been rendered unnecessary
with advancements in the ability to collect robust data from
the wrist and transmit it wirelessly. These improvements have
greatly increased our capacity for measuring on-the-go.
The E4 collects signals produced by the body's electrodermal
activity (EDA), skin temperature (SKT), and blood
volume pulse (BVP) and provides a feature calculated from
BVP as a continuous signal: heart rate (HR) [11]. These signals
have been studied in related research [10], [20]. Based on previous
research [9],[12], we extract several features from these
four signals, as described further in the feature extraction process
section.
Analyzing physiological correlates requires the use of machine
learning algorithms that can detect changes in these
correlates with high levels of accuracy. Direct correlation analysis
that detects only linear relationships between signals and
affect, and static rules defining when a threshold is crossed,
limit the type of patterns that can define an affective state of
interest [17]. However, machine learning algorithms can recognize
non-linear patterns and leverage high-dimensional
dataspaces [9],[10],[13],[17]. An effective algorithm would:
predict between affective states of interest with accuracy better
than chance; have an area under the curve (AUC) as close
to one as possible; and have a variance between training and
testing results as close to zero as possible. The purpose of the
current case study was to evaluate algorithms in the context of
RAI to determine their potential efficacy in determining when
students are attentive during RAI-related tasks. We addressed
the following two research questions: Do physiological signals
40
indicate different affective states of children with ASD during
RAI? And How accurate are machine learning algorithms,
built from physiological signals, at matching expert coders
when differentiating between levels of attention by children
with ASD?
In this investigation, we sought to evaluate algorithms
in the context of RAI to determine their potential efficacy in
determining when students with ASD are attentive during
RAI-related tasks. We calculated features from physiological
signals collected by an E4 and compared the performance of
four machine learning algorithms commonly used in affective
computing. Our findings indicated the E4 data were useful for
categorizing states of attentiveness in children with ASD.
Method
Participants and Setting
We recruited participants through a university-affiliated autism
clinic that regularly conducted social skills groups for
children with ASD. We obtained parental consent and participant
assent prior to the study. Two children with ASD (i.e.,
Cody and Max), age 11 years, participated in the investigation.
Both participants were diagnosed as having ASD, using the
Autism Diagnosis Observation Schedule-2, and identified to have
difficulties in initiating and maintaining social interactions.
Both participated in a social skills group at the autism clinic.
We observed and collected data samples during 5-min observations
of social skills group activities that took place over
several weeks. Due to the availability of only two E4s, we only
included two participants in the study. Specifically, we collected
data during a pilot evaluation of RAI and unstructured
social interaction probes. During RAI, the children with ASD
were seated in front of an NAO robot (Fig. 1) and directed to
engage in scripted interactions. During the social interaction
probes (SIP), the children with ASD were seated around a table
and directed to talk with each other.
Fig. 1. An NAO robot sits on a cart, ready to interact with children during the
social skills intervention.
IEEE Instrumentation & Measurement Magazine
May 2023
Instrumentation & Measurement Magazine 26-3

Table of Contents for the Digital Edition of Instrumentation & Measurement Magazine 26-3
