IEEE Computational Intelligence Magazine - May 2021 - 51

that jointly predicts health, job-performance, and psychometric
variables of individuals in a global manner.
The present work is a broad and personalized analysis using
instruments from the longitudinal Tesserae Project [24]. An initial analysis based solely on job performance was presented in
[42], which reported a model to differentiate low from high
job performance but did not estimate the specific job performance score directly. [42] reported predictions of the daily battery. In contrast, our analysis is done on the single initial battery
of twelve standardized tests not only of job performance, but of
all other measures as well.
IV. Data Collection and Description

From Fall 2017 to Summer 2018 we recruited 757 individuals
working in knowledge fields in the US as part of a large-scale
longitudinal research study. We collected data from these participants for a period of one year starting from January 2018.
Individuals' participation in the study was voluntary and those
who participated received a monetary incentive to stay in the
study and comply with the data-collection protocols. This
monetary compensation varied according to levels of compliance and was allocated throughout the year of study. The monetary compensation for participants was also specific for one of
the companies, per the rules of the company.
Our project was conducted in accordance with the Institutional Review Board of the University of Notre Dame (under
protocol number 17-05-3870) and similar authorities of all the
institutions involved. All participants provided written
informed consent prior to taking part in the study. No Personal
Identifiable Information (PII) was shared.
To handle the heterogeneity of our dataset, a subset of
participants was selected from each cohort in Table III for
external validation and was not considered during development of our models in order to prevent bias and data leakage. The remaining 554 participants came from various
organizations in the USA and can be grouped into five
cohorts, as shown in Table III. Another source of heterogeneity, particularly at the job-performance level, comes from
the participants' roles. A total of 254 participants self-reported holding a supervisory whereas 297 reported a nonsupervisory role; 3 participants declined to mention their
role within their companies.
The data collection protocols could be classified into
two stages: 1) an initial set of surveys used to collect the
initial battery of ground truth variables and social media
data; and 2) daily data-gathering of data streams from various sensors (daily varying predictors). We analyzed the initial ground truth battery using the daily sensor data streams
and social media.
A. Data Sources-Sensing Streams

In order to model individuals' behaviors and physical attributes,
we selected multiple modalities that unobtrusively collect physiological, psychological, behavioral, and physical states of individuals; their offline and online interactions; their phone, social

media activity, and workplace routines; and health and wellbeing both at work and at home. Specifically, we used a wearable to capture an individual's physical and physiological state.
In order to capture the context of an individual's actions we
used a phone agent (app) and proximity beacons that allowed
us to identify the individuals' relative locations (home/work)
during the day. Finally, we capture higher level information
using social media, together with the wearable and phone agent
data, provided insights about a person's psychological states. All
data was de-identified to protect the participants privacy. In
addition to raw features, we considered features derived from
the sensors.
Wearable: Garmin Vivosmart 3. This wristband is a commercial smart wristband (a wearable device) that is widely used as
a fitness, activity, and well-being monitoring device. The
device collects physical/physiological data, (e.g., heartrate, step
count, number of floors climbed, calories burned, physical
activity such as running, and walking), sleep quality data (e.g.,
sleep staging, duration), and psychological data (e.g., stress-
which is based on physical signals such as heart rate). The
wearable was paired via Bluetooth with Connect, a Garmin
App that participants installed on their phones. The wearable
was also paired with an app we developed for our study (see
PhoneAgent below). Both apps collected data from the wearable which was transferred to our collecting servers and into
databases that anonymized the data. We computed daily summaries from each of the signals collected.
App: PhoneAgent. We created an app (the PhoneAgent) for
both iOS and Android devices. The app ran in the background
and periodically collected data, saving it temporarily as JSON
files that were later transmitted to servers when the phone was
connected to Wi-Fi. The data collected by our app included
location, physical activity (walking, bicycling, driving, etc.),
phone usage (e.g., screen lock/unlock) and ambient light levels.
Our app also connected to the wearable and the beacons
(described below) via Bluetooth. The PhoneAgent streamed
data directly from the wearable, which allowed for more finegrained and real-time data collection compared to Garmin's
Connect app (e.g., beat-to-beat interval in the PhoneAgent
compared to average heart rate every minute from Connect).
Our app collected the following wearable generated timeseries: heart rate (HR), steps, floors climbed, calories burned,
and stress levels. From the beacons (see below), our app collected information about the proximity of an individual (through a
key-chain beacon and backpack beacon) to either of the fixed

TABLE III Participants per cohort used for modeling.
COHORT

# PARTICIPANTS

1. Multinational Consultancy Company

217

2. Multinational Technology Company

138

3. Small Software Company

4. Various Smaller Companies

147

5. Local University

MAY 2021 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE

IEEE Computational Intelligence Magazine - May 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - May 2021