IEEE Computational Intelligence Magazine - May 2021 - 53

We experimented with various time resoluWe derived features from the time series. This
tions to derive the summary statistics, as the distributions may have non-linear relations that may not
included higher order networks to represent nonfully capture the individual's behavior. We report
Markovian patterns.
predictions for individuals with at least two weeks
of data. We constructed HON representations of
people's behaviors through the heart rate and stress time series as
ables. As we detailed in Section IV-A, the data sources
we describe in Section V.
included social media, Garmin wearable, phone agent, and
The predictors are highly heterogeneous due to the multibeacon data. Additionally, we computed heart-rate variability
modal nature of our dataset. This made it prudent to apply
and used it as a separate stream. Furthermore, we constructed
ensemble-learning strategies. Another source of heterogeneity
a HON based on heart rate and wearable/sensor stress meaand noise in the features was due to the compliance of the parsures as detailed below. Within each model, a set of candidate
ticipants, the quality of the data transfer, and missing data.
models are trained per ground truth variable as outlined in
our model's schema in Figure 1. The components of the
ensemble are supervised techniques (regression and classificaC. Missing Data and Data Difficulties
tion) as detailed next. In order to deal with the complexity
In addition to the heterogeneity of the data sources, the main
of the data as well as the missingness, we considered the folchallenge of building predictive models with our data set was
lowing: a) the ensemble components that identify both linear
caused by missing values. The data sources most affected by feaand non-linear partitions and regressions, b) the pre- and
ture missingness, i.e., missing values of specific predictors, were
post-processing that ensure generality and avoid outliers, c)
the wearable and the PhoneAgent. In particular, missingness in
the feature selection that eliminates redundant dimensions
the latter was critical as the PhoneAgent was used to collect
and selects relevant features, d) a higher-order representation
data from the wearable and the beacons.
of temporal data that extracts non-Markovian patterns (longPhoneAgent. Missing data from the PhoneAgent was mostly
term temporal dependencies), e) imputations, both at the
due to technical issues. In particular, participants had a variety
feature and modality level, f) a fusion strategy for the various
of phone models with different operating systems (and vermodalities, and g) an algorithm to coordinate the model
sions) and capabilities. This variability in devices made it diffiselection framework.
cult to provide user support. Also, some adjustments were
needed because both the Garmin platform and the beacons did
not record data properly in some iOS versions.
A. Design of Components
Wearable. Missingness was due to loss, failure (e.g., did not
Our design goal was to automate the discovery of variable relahold charge, did not charge at all, data did not sync, unusual
tions and data separability for linear, multicollinear, and nonlinreport of floors climbed, inability to connect to the phone,
ear relations. In each case we consider low and high
inadequate sleep tracking on public transport.), or damage to
dimensional cases. Thus, we considered the following regression
the device (e.g., strap, screen) or charger. One participant
methods as candidates for the components: linear regression
reported an allergic reaction to the nickel in the buckle.
(low-dimensional cases), linear regression with L2-norm (mulSocial Media. This stream presented two challenges: not all
ticollinearities cases), linear regression with built-in cross-valiindividuals had Facebook accounts and the level of engagedation with L 2-norm (high dimensional multicollinear
ment of individuals in their online profiles varied greatly.
relations), lasso model with least angle regression (high-dimenBeacons. Some participants placed home beacons at the
sional linear cases), Bayesian ridge regression (high dimensional
work place and vice versa. This made extracting meaningful
cases), support vector regressor (SVR) with either linear, radial
features related to location challenging. Additionally, noisy data
basis function, or polynomial kernel (for linear and non-linear
also presents some challenges. Bluetooth signal strength can be
high-dimensional relations). Finally, decision trees (CART), and
affected by certain objects and their properties (e.g., number of
random forest regression were used for non-linear relations.
walls and the materials used in construction). Beacons (and the
The selection of the optimal technique and corresponding feawearable) can also be affected by noise because sleep during
tures was done using cross-validation, as detailed below, which
the day may not be reliably detected.
allows us to pick the best performer per ground truth variable.
Finally, in addition to feature missingness, a major challenge
The best performers were then used for training and predicis full-modality missingness, i.e., participants with information
tion. Likewise, we used classification counter-parts for linear
missing for the entire modality, as in the case of social media,
and non-linear separability and high vs. low-dimensional probwhere no data was available for several participants. In such
lems. Specifically, we considered: k nearest-neighbors, linear
cases we used group imputation methods as detailed next.
support vector machine, support vector machine with radialbasis function, decision trees, and random forest. At a lower
V. Joint Prediction Model
level, various intermediate steps were performed: t-ransformation,
We developed an ensemble learning method for joint predicmapping, dimensionality reduction, fusion of sub-datasets, and
tion of the physical, psychological, and job-performance varifeature selection.

MAY 2021 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE

IEEE Computational Intelligence Magazine - May 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - May 2021