IEEE Computational Intelligence Magazine - May 2021 - 57

comparable to theory-driven (survey-based) predictions. To create a
Anxiety
baseline theoretical model, we use
Vocabulary
the distributions of each variable
Abstraction
and take the expected value of the
training folds to estimate the values
Sleep
for the test fold. Table V shows the
Openness
symmetric mean absolute percentOCB
age error (SMAPE) for each of the
Neuroticism
variables and the two models. This
ITP
table demonstrates that using only
IRB
sensor-based estimates (our framework) leads to estimations with
Physical Activity
smaller errors compared to a baseTobacco
line that is based on surveys. The
Extraversion
sensor-based predictions are entireOrganizational D.
ly based on wearable sensor and
Interpersonal D.
social media data; no survey or
Conscientiousness
demographic data that could otherwise facilitate identification of patAlcohol
terns based on personal traits were
Agreeableness
used. Our method shows improved
Positive Affect
performance over the baseline.
Negative Affect
2) Job Perfor mance-Improvement
-0.1
0
0.1
0.2
0.3
0.4
0.5
Assessment Over Participant-Oriented
Baseline: We evaluate the interplay
of psychological and job-performance variables. Thus, we compare FIGURE 4 Model Reliability-cross validation: Kendall's x confidence interval of the socio-psycho-physiological variable predictions. Distribution 5-fold cv.
the x score accounted for by sensor-derived estimates, beyond what
is accounted for by two known predictors of job perforTABLE VI Model reliability-External validation.
mance, personality ([11], [12], [14], [15], [30], [54]) and
VARIABLE
MIN
MAX
MEAN
cognitive ability [53], [142]. This baseline is then compared
Vocabulary
0.00
0.26
0.10
with the estimation of job performance using our sensorbased framework. To assess the relevance of our estimations,
Abstraction
0.00
0.27
0.11
we considered the x- score of each job performance variExtraversion
0.02
0.36
0.19
able when predicted with the baseline and with the sensorAgreeableness
-0.09
0.22
0.07
derived estimates. Table VIII shows the expected x- score
Conscientiousness
0.05
0.31
0.15
for both the theoretical baseline and our framework. This
Neuroticism
-0.19
0.18
0.00
table shows our sensor-based model has a higher expected
Openness
-0.11
0.305
0.14
performance for all the variables. However, in order to fully
Positive Affect
-0.07
0.32
0.16
validate this result, we also performed a comparative analysis of the full set of predictions using a 5-fold cross-validaNegative Affect
-0.16
0.18
0.01
tion approach. Then, we built the distribution of differences
Anxiety
0.00
0.31
0.14
of estimations based on our model minus the estimations
Alcohol
0.22
0.49
0.37
based on theory. The mode of the distributions of all the
Tobacco
-0.13
0.00
-0.04
job performance variables lie in the region T x 2 0 where
Physical Activity
0.20
0.52
0.37
T x is the difference of x scores. Thus, our model performs
Sleep
0.03
0.35
0.20
better than the theory-based estimation in the majority
of cases.
the predictions of other constructs (main diagonal = -). As
3) Discriminant Validity: We examined correlations between
shown in Table VII, the model has good discriminant validconstructs and their predictions. The discriminant validity
ity with correlations in the range 6- 0.21, 0.2@ .
is shown in Table VII. The objective was to verify whether
our model sufficiently discriminated the constructs which
4) Model Reliability: Figure 4 shows the distribution of x
is signaled by low inter-correlations among constructs and
scores for each of the variables estimated. We build this

MAY 2021 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE

IEEE Computational Intelligence Magazine - May 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - May 2021