IEEE Computational Intelligence Magazine - May 2021 - 59

baselines. Despite the wealth of sources and features we used,
predicting job performance and psychological constructs is a
harder task than predicting physical well-being (alcohol consumption, sleep, etc.). While predicting job performance is a
difficult task, most physical variables were predicted well. The
predictions in our 5-fold validations and in the externally validated sample were comparable.
Our contribution is three-fold. First, we identified strategies
for integrating highly heterogeneous data without curation,
and thus, maintained the data integrity. Second, we analyzed
the different challenges presented by non-curated data with a
systematic feature mining approach. Third, we created a benchmark for predictive tasks by leveraging the identified challenges
of the real noisy or incomplete multi-modal high-dimensional
data to create a comprehensive prediction and assessment of
well-being: physical, psychological, and work-place well-being
characteristics of individuals. Development of effective affectcomputing systems must include the century-long research on
emotion created by psychology. Thus, we contributed in this
area as well. Our work's realistic assessment of machine learning
applied to performance prediction could also provide benefits
for mitigating bias [147]. Our work can be used towards the
creation of more objective measures of job performance, and as
a realistic and sound baseline for analysis.
Acknowledgment

This research is based upon work supported in part by the
Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), via
IARPA Contract No. 2017-17042800007. The views and conclusions contained herein are those of the authors and should
not be interpreted as necessarily representing the official policies, either expressed or implied, of ODNI, IARPA, or the U.S.
Government. The U.S. Government is authorized to reproduce
and distribute reprints for governmental purposes notwithstanding any copyright annotation therein.
This article has supplementary downloadable material available at https://doi.org/10.1109/MCI.2021.3061877, provided
by the authors.
References

[1] A. Abbas, M. Ali, M. U. S. Khan, and S. U. Khan, " Personalized healthcare
cloud services for disease risk assessment and wellness management using social media, " Pervasive Mobile Comput., vol. 28, pp. 81-99, 2016. doi: 10.1016/j.pmcj.2015.
10.014.
[2] R. Z. Goetzel et al., " Do workplace health promotion (wellness) programs work? "
J. Occupat. Environ. Med., vol. 56, no. 9, pp. 927-934, 2014. doi: 10.1097/JOM.
0000000000000276.
[3] S. G. Aldana, R. M. Merrill, K. Price, A. Hardy, and R. Hager, " Financial impact of
a comprehensive multisite workplace health promotion program, " Prevent. Med., vol. 40,
no. 2, pp. 131-137, 2005. doi: 10.1016/j.ypmed.2004.05.008.
[4] X. Yang, C. Ge, B. Hu, T. Chi, and L. Wang, " Relationship between quality of life
and occupational stress among teachers, " Public Health, vol. 123, no. 11, pp. 750-755,
2009. doi: 10.1016/j.puhe.2009.09.018.
[5] A. Sano, P. Johns, and M. Czerwinski, " Designing opportune stress intervention delivery timing using multi-modal data, " in Proc. 7th Int. Conf. Affective Comput. Intell. Interaction (ACII), 2017, pp. 346-353. doi: 10.1109/ACII.2017.8273623.
[6] J. M. Smyth et al., " Everyday stress response targets in the science of behavior change, "
Behav. Res. Ther., vol. 101, pp. 20-29, 2018. doi: 10.1016/j.brat.2017.09.009.
[7] M. Quante et al., " Seasonal and weather variation of sleep and physical activity in
12-14-year-old children, " Behav. Sleep Med., vol. 17, no. 4, pp. 398-410, 2019. doi:
10.1080/15402002.2017.1376206.

[8] T. A. Wright and R. Cropanzano, " Psychological well-being and job satisfaction as
predictors of job performance, " J. Occupat. Health Psychol., vol. 5, no. 1, p. 84, 2000. doi:
10.1037/1076-8998.5.1.84.
[9] D. S. Chiaburu, I.-S. Oh, C. M. Berry, N. Li, and R. G. Gardner, " The five-factor
model of personality traits and organizational citizenship behaviors: A meta-analysis, " J.
Appl. Psychol., vol. 96, no. 6, p. 1140, 2011. doi: 10.1037/a0024004.
[10] D. Kamdar and L. Van Dyne, " The joint effects of personality and workplace social
exchange relationships in predicting task performance and citizenship performance, " J.
Appl. Psychol., vol. 92, no. 5, p. 1286, 2007. doi: 10.1037/0021-9010.92.5.1286.
[11] M. R. Barrick and M. K. Mount, " The Big Five personality dimensions and job
performance: A meta-analysis, " Personnel Psychol., vol. 44, no. 1, pp. 1-26, 1991. doi:
10.1111/j.1744-6570.1991.tb00688.x.
[12] M. R. Barrick, M. K. Mount, and T. A. Judge, " Personality and performance at the
beginning of the new millennium: What do we know and where do we go next?, " Int. J.
Sel. Assess., vol. 9, nos. 1&2, pp. 9-30, 2001. doi: 10.1111/1468-2389.00160.
[13] H. J. Eysenck, Ed., A Model for Intelligence. Springer-Verlag, 1982.
[14] J. F. Salgado, " The five factor model of personality and job performance in the European Community, " J. Appl. Psychol., vol. 82, no. 1, p. 30, 1997. doi: 10.1037/00219010.82.1.30.
[15] R. P. Tett, D. N. Jackson, and M. Rothstein, " Personality measures as predictors of
job performance: a meta-analytic review, " Personnel Psychology, vol. 44, no. 4, pp. 703-742,
1991. doi: 10.1111/j.1744-6570.1991.tb00696.x.
[16] J. Andreu-Perez, D. Leff, H. M. Ip, and G.-Z. Yang, " From wearable sensors to smart
implants - toward pervasive and personalized healthcare, " IEEE Trans. Biomed. Eng.,
vol. 62, no. 12, pp. 2750-2762, 2015. doi: 10.1109/TBME.2015.2422751.
[17] F. Schaule, J. O. Johanssen, B. Bruegge, and V. Loftness, " Employing consumer
wearables to detect office workers' cognitive load for interruption management, " Proc.
ACM Interactive, Mobile, Wearable Ubiquitous Technol., vol. 2, no. 1, pp. 32, 2018. doi:
10.1145/3191764.
[18] D. O. Olguín, P. A. Gloor, and A. S. Pentland, " Capturing individual and group
behavior with wearable sensors, " in Proc. AAAI Spring Symp. Human Behavior Modeling,
SSS, 2009, vol. 9, pp. 68-74.
[19] M. J. Salganik et al., " Measuring the predictability of life outcomes with a scientific
mass collaboration, " Proc. Nat. Acad. Sci., vol. 117, no. 15, pp. 8398-8403, 2020. doi:
10.1073/pnas.1915006117.
[20] T. A. Judge and C. P. Zapata, " The person-situation debate revisited: Effect of situation strength and trait activation on the validity of the Big Five personality traits in
predicting job performance, " Acad. Manage. J., vol. 58, no. 4, pp. 1149-1179, 2015. doi:
10.5465/amj.2010.0837.
[21] C. Viswesvaran and D. S. Ones, " Perspectives on models of job performance, " Int. J.
Sel. Assess., vol. 8, no. 4, pp. 216-226, 2000. doi: 10.1111/1468-2389.00151.
[22] C. H. Mallinckrodt et al., " Assessing and interpreting treatment effects in longitudinal clinical trials with missing data, " Biol. Psychiatr., vol. 53, no. 8, pp. 754-760, 2003. doi:
10.1016/S0006-3223(02)01867-X.
[23] G. Molenberghs et al., " Analyzing incomplete longitudinal clinical trial data, " Biostatistics, vol. 5, no. 3, pp. 445-464, 2004. doi: 10.1093/biostatistics/kxh001.
[24] S. M. Mattingly et al., " The Tesserae project: Large-scale, longitudinal, in situ, multimodal sensing of information workers, " in Proc. Extended Abstracts CHI Conf. Human
Factors Comput. Syst., 2019, p. CS11. doi: 10.1145/3290607.3299041.
[25] A. Madan, S. T. Moturu, D. Lazer, and A. S. Pentland, " Social sensing: Obesity,
unhealthy eating and exercise in face-to-face networks, " in Proc. Wireless Health 2010,
2010, pp. 104-110.
[26] N. J. Yuan, F. Zhang, D. Lian, K. Zheng, S. Yu, and X. Xie, " We know how you
live: Exploring the spectrum of urban lifestyles, " in Proc. 1st ACM Conf. Online Soc. Netw.,
2013, pp. 3-14.
[27] F. Calabrese, M. Diao, G. D. Lorenzo, J. Ferreira, Jr, and C. Ratti, " Understanding individual mobility patterns from urban sensing data: A mobile phone trace example, " Transportation Res. C, Emerg. Technol., vol. 26, pp. 301-313, 2013. doi: 10.1016/j.
trc.2012.09.009.
[28] B. Mariani, M. C. Jiménez, F. J. G. Vingerhoets, and K. Aminian, " On-shoe wearable
sensors for gait and turning assessment of patients with Parkinson's disease, " IEEE Trans.
Biomed. Eng., vol. 60, no. 1, pp. 155-158, 2013. doi: 10.1109/TBME.2012.2227317.
[29] L. J. Williams and S. E. Anderson, " Job satisfaction and organizational commitment as predictors of organizational citizenship and in-role behaviors, " J. Manage., vol. 17,
no. 3, pp. 601-617, 1991. doi: 10.1177/014920639101700305.
[30] M. A. Griffin, A. Neal, and S. K. Parker, " A new model of work role performance:
Positive behavior in uncertain and interdependent contexts, " Acad. Manage. J., vol. 50,
no. 2, pp. 327-347, 2007. doi: 10.5465/amj.2007.24634438.
[31] S. Fox, P. E. Spector, A. Goh, K. Bruursema, and S. R. Kessler, " The deviant citizen: Measuring potential positive relations between counterproductive work behaviour
and organizational citizenship behaviour, " J. Occup. Organizat. Psychol., vol. 85, no. 1,
pp. 199-220, 2012. doi: 10.1111/j.2044-8325.2011.02032.x.
[32] R. J. Bennett and S. L. Robinson, " Development of a measure of workplace deviance, " J. Appl. Psychol., vol. 85, no. 3, pp. 349, 2000. doi: 10.1037/0021-9010.85.3.349.
[33] W. C. Shipley, C. P. Gruber, T. A. Martin, and A. M. Klein, Shipley-2 Manual, Los
Angeles, CA: Western Psychological Services, 2009.
[34] C. J. Soto and O. P. John, " The next Big Five Inventory (BFI-2): Developing and
assessing a hierarchical model with 15 facets to enhance bandwidth, fidelity, and predictive power, " J. Personality Soc. Psychol., vol. 113, no. 1, p. 117, 2017. doi: 10.1037/
pspp0000096.
[35] D. Watson and L. Clark, The PANAS-X: Manual for the Positive and Negative Affect
Schedule-Expanded Form. Univ. of Iowa, 1994.

MAY 2021 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE

59
https://www.doi.org/10.1109/MCI.2021.3061877
IEEE Computational Intelligence Magazine - May 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - May 2021
