IEEE Systems, Man and Cybernetics Magazine - October 2021 - 9

the microexpressions on the face leading to repeated behaviors
and sounds can be used as inputs to a DL algorithm to
classify vocal stereotypy among other behaviors. We have
found the Microsoft Azure Kinect camera to be a well-rounded
sensor to record body-tracking data, red-green-blue,
infrared, and depth videos, and also stereo audio data. Additionally,
we found the Empatica E4 wearable sensor to be
reliable to study sleep disorders using heart rate, electrodermal
activity, and other sensory data, as illustrated in Figure 4.
Integration of data collected via behavioral sensing can lead
to deeper understanding of the triggers of problem behavior,
such as poor sleep the night before. It can also help to distinguish
behaviors triggered by environmental events versus
those rooted in biological processes, and facilitate early identification
of behaviors via precursors such as heart rate. Precise
understanding of behavioral triggers is essential to
ensuring effective and efficient treatment.
Various other actions, such as pacing in a repeated
circular motion, body rocking, back-and-forth movement
of fingers, tapping objects repeatedly, and so on
Table 1. A summary of common behavioral challenges, their descriptions,
and possible measurements using sensors.
Behavioral
Challenge
Communication
Delays
Description
Children with neurodevelopmental disabilities, such as
ASD, often require communication interventions. Additionally,
approximately 30% of individuals with ASD do
not fully acquire spoken communication [22].
Possible Sensors and Measurements
Vocal communication could be more precisely evaluated
using high-tech data collection, which could promote
more accurate and reliable intervention procedures. Automated
data collection could be embedded in speechgenerating
device software along with isolated teaching
platforms to teach the use of these alternative communication
modes and promote data-based decision making.
Problem Behavior
End result of a chain of events that usually triggers
(reactive, impulsive) during interactions with social and
physical environments often accompanied by anger
or frustration. Problem behavior, such as aggression,
can be toward self, others, or the environment, while
verbal aggression is usually toward others. Understanding
the underlying conditions and patterns are very
important for BAs to help reduce problem behavior.
Agitation
Vocal Stereotypy
Sleep Disorder
Restless or hyperactive stages, usually due to unrecognized
physical or sensory discomfort, which could lead
to anxiety and verbal or physical aggression.
Persistent repetitions of words, phrases, or sounds without
contextual or functional meaning in the current
social setting.
Trouble falling asleep or staying asleep for long periods of
time. It could affect academic and social behaviors due
to mood problems, memory, concentration, and learning
problems, sluggish reaction time, and so on caused by
disrupted sleep patterns, insomnia, or sleep apnea [23].
Social Delay
Difficulties with understanding social rules and interactions.
Can be related to communication delay and
engagement in problem behavior but may also be
present in individuals without communication delays.
Problem behavior such as biting, hitting, and kicking
could be tracked using body-tracking sensors and
algorithms [11]. Vocal aggression such as screaming,
shouting, use of foul language, and so on could be
found by recording the speech and studying them using
intelligent algorithms.
Physiological data from wearable devices and EEG
systems could help understand the long-term triggers
and etiological factors leading to agitation.
The vocal behavior could be found by recording the
speech and studying them using intelligent algorithms
in a functional analysis experimental setting.
Wearable sleep-staging sensors or sleep-sensing mats
could identify sleep patterns before and after behavioral
therapies to study impact of the therapies in sleep
and to design therapies that could improve sleep.
Specially designed games on AR/VR environments
could teach social skills and provide a structured environment
for practicing vital social interactions.
Figure 2. Tracking aggressive behavior using a body-tracking AI algorithm is illustrated. Here, forceful
movements to the head can be classified as banging the head using a temporal action indexing module [11],
as described in our previous research.
October 2021 IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE
9

IEEE Systems, Man and Cybernetics Magazine - October 2021

Table of Contents for the Digital Edition of IEEE Systems, Man and Cybernetics Magazine - October 2021