Computational Intelligence - November 2016 - 24
Regularized Multivariate
Analysis Framework
for Interpretable
High-Dimensional
Variable Selection
Sergio Muñoz-Romero
Department of Signal Processing and Communications,
Universidad Rey Juan Carlos, Madrid, SPAIN
Vanessa Gómez-Verdejo and Jerónimo Arenas-García
Department of Signal Processing and Communications,
Universidad Carlos III de Madrid, SPAIN
m
ultivariate Analysis (MVA) comprises a collection of tools that play a fundamental role in statistical data analysis. These techniques have become
increasingly popular since the proposal of Principal Component Analysis
(PCA) in 1901 [1]. PCA was proposed as a simple and efficient way to
reduce data dimension by projecting the data over the largest variance directions. As
illustrated in Fig. 1, PCA learns from a given dataset a set of projection vectors, so
that data can be represented in a low-dimensional space that preserves the directions
of the input space where the data shows the largest variance. A typical example to
illustrate PCA is face recognition, where the projection vectors are known as eigenfaces [2]. Nevertheless, PCA has been used in many other applications, and can
indeed be considered as one of the most widely-used tools for feature extraction.
Other MVA algorithms have emerged that are especially suited to supervised
learning tasks (e.g., in regression and classification). In these problems, the goal is not
just to represent the input data as efficiently as possible, but it actually becomes of
major importance to keep the directions of the input space that are more highly
correlated with the label information. This is the case of algorithms such as Canonical Correlation Analysis (CCA) [3], Partial Least Squares (PLS) approaches [4], [5],
and Orthonormalized PLS (OPLS) [6]. Consider for instance a toy classification
problem in Fig. 2. In this problem, the direction of the maximum variance extracted
by PCA (left subplot) results in overlapping distributions of the two classes along this
image licensed by ingram publishing
I. Introduction
Digital Object Identifier 10.1109/MCI.2016.2601701
Date of publication: 12 October 2016
24
IEEE ComputatIonal IntEllIgEnCE magazInE | novEmbEr 2016
1556-603x/16©2016IEEE
Table of Contents for the Digital Edition of Computational Intelligence - November 2016
Computational Intelligence - November 2016 - Cover1
Computational Intelligence - November 2016 - Cover2
Computational Intelligence - November 2016 - 1
Computational Intelligence - November 2016 - 2
Computational Intelligence - November 2016 - 3
Computational Intelligence - November 2016 - 4
Computational Intelligence - November 2016 - 5
Computational Intelligence - November 2016 - 6
Computational Intelligence - November 2016 - 7
Computational Intelligence - November 2016 - 8
Computational Intelligence - November 2016 - 9
Computational Intelligence - November 2016 - 10
Computational Intelligence - November 2016 - 11
Computational Intelligence - November 2016 - 12
Computational Intelligence - November 2016 - 13
Computational Intelligence - November 2016 - 14
Computational Intelligence - November 2016 - 15
Computational Intelligence - November 2016 - 16
Computational Intelligence - November 2016 - 17
Computational Intelligence - November 2016 - 18
Computational Intelligence - November 2016 - 19
Computational Intelligence - November 2016 - 20
Computational Intelligence - November 2016 - 21
Computational Intelligence - November 2016 - 22
Computational Intelligence - November 2016 - 23
Computational Intelligence - November 2016 - 24
Computational Intelligence - November 2016 - 25
Computational Intelligence - November 2016 - 26
Computational Intelligence - November 2016 - 27
Computational Intelligence - November 2016 - 28
Computational Intelligence - November 2016 - 29
Computational Intelligence - November 2016 - 30
Computational Intelligence - November 2016 - 31
Computational Intelligence - November 2016 - 32
Computational Intelligence - November 2016 - 33
Computational Intelligence - November 2016 - 34
Computational Intelligence - November 2016 - 35
Computational Intelligence - November 2016 - 36
Computational Intelligence - November 2016 - 37
Computational Intelligence - November 2016 - 38
Computational Intelligence - November 2016 - 39
Computational Intelligence - November 2016 - 40
Computational Intelligence - November 2016 - 41
Computational Intelligence - November 2016 - 42
Computational Intelligence - November 2016 - 43
Computational Intelligence - November 2016 - 44
Computational Intelligence - November 2016 - 45
Computational Intelligence - November 2016 - 46
Computational Intelligence - November 2016 - 47
Computational Intelligence - November 2016 - 48
Computational Intelligence - November 2016 - 49
Computational Intelligence - November 2016 - 50
Computational Intelligence - November 2016 - 51
Computational Intelligence - November 2016 - 52
Computational Intelligence - November 2016 - 53
Computational Intelligence - November 2016 - 54
Computational Intelligence - November 2016 - 55
Computational Intelligence - November 2016 - 56
Computational Intelligence - November 2016 - 57
Computational Intelligence - November 2016 - 58
Computational Intelligence - November 2016 - 59
Computational Intelligence - November 2016 - 60
Computational Intelligence - November 2016 - 61
Computational Intelligence - November 2016 - 62
Computational Intelligence - November 2016 - 63
Computational Intelligence - November 2016 - 64
Computational Intelligence - November 2016 - 65
Computational Intelligence - November 2016 - 66
Computational Intelligence - November 2016 - 67
Computational Intelligence - November 2016 - 68
Computational Intelligence - November 2016 - 69
Computational Intelligence - November 2016 - 70
Computational Intelligence - November 2016 - 71
Computational Intelligence - November 2016 - 72
Computational Intelligence - November 2016 - Cover3
Computational Intelligence - November 2016 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202311
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202308
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202305
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202302
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202211
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202208
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202205
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202202
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202111
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202108
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202105
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202102
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202011
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202008
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202005
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202002
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201911
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201908
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201905
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201902
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201811
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201808
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201805
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201802
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter12
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall12
https://www.nxtbookmedia.com