Computational Intelligence - November 2016 - 21
MMSE
MMSE
3. performance Comparison
From a Bayesian perspective it can be interpreted
In Figure 2, the average reconstruction error
(NMSE) with increasing support cardinality
that for each algorithm a fixed prior distribution has
is plotted for the case where the non-zero
been employed. But in real life that assumed prior
coefficients are from a zero mean, unit varidistribution may be significantly different from the
ance, Gaussian distribution. It is evident that
our proposed adaptive approach gives the
true distribution of the data.
best performance in this case with Reweighted , 2 minimization approach being a very
distributional parameters our unified MAP estimation
close competitor. It is worth noting that Reweighted , 2
framework leads to several popular SSR algorithms, specifiworks much better compared to the other two methods
cally reweighted algorithms available in the literature. Based
with fixed distributional parameters, and the reason being
on this result, we proposed an adaptive framework where
the heuristic update of e, which helps it to get away from
the distributional parameters of GT have not been fixed
local minima. Hence, e update in Reweighted , 2 is absobeforehand, and adapted based on the measurement over
lutely necessary as we have found out for fixed e this algoiterations. Our extensive empirical results also show the effirithm's performance decreases significantly. This heuristic
cacy of this adaptive approach over other MAP estimation
update of e falls into the adaptive paradigm but our probased SSR algorithms.
posed approach provides a systematic approach to adapt
both the distributional parameters.
In Figure 3, the average reconstruction error (NMSE)
with increasing support cardinality is plotted where the
non-zero coefficients are generated from a Laplace distri0.35
bution with variance 1. Again, the empirical superiority of
0.30
our proposed adaptive approach over other algorithms is
0.25
evident from Figure 3. An interesting point to note here, is
0.20
that Reweighted , 1 with fixed e matches the performance
of Reweighted , 2 with e update and the reason could be
0.15
that true distribution of the non-zero coefficients and
0.10
the assumed prior for Reweighted , 1 have the similar
0.05
tail behavior.
25
30
35
40
45
50
Finally, we repeat the same set of experiments where the
k
non-zero coefficients follow a sub Gaussian distribution, i.e.
Reweighted /2
Adaptive /p
Uniform ±1 random spikes, and the plot of the average reconReweighted /1
LASSO (/1)
struction error (NMSE) with increasing support cardinality is
shown in Figure 4. The performance improvement of the proFigure 2 recovery performance with Gaussian distributed non-zero
posed approach over other reweighted algorithms is significant
coefficients.
compared to the previous two cases.
In Figure 5 we visualize the adaptation of the distributional
parameters step. We consider a case where k = 30 and the nonzero entries have been randomly drawn from zero mean and
0.24
unit variance Gaussian distribution. On top, the absolute value
0.22
of the true non-zero coefficients have been plotted. Next two
0.20
figures represent the learned values of q and p after learning
0.18
them using our proposed algorithm. We see how the adaptation
0.16
0.14
step helps us to learn the corresponding distributional parame0.12
ters which plays a key role in modeling the tail nature of the
0.10
sparsity inducing prior distributions.
0.08
0.06
V. Conclusion and Discussion
In this paper, we formulated the SSR problem from a Bayesian perspective and discussed a generalized scale mixture
family: PESM in detail. We analyzed the tail behavior of the
GT class of distributions, which is a member of PESM family, and discussed when a GT distribution will be suitable to
model sparsity. We also showed, how by choosing specific
25
30
35
40
45
50
k
Reweighted /2
Reweighted /1
Adaptive /p
LASSO (/1)
Figure 3 recovery performance with super Gaussian (laplace)
distributed non-zero coefficients.
novEmbEr 2016 | IEEE ComputatIonal IntEllIgEnCE magazInE
21
Table of Contents for the Digital Edition of Computational Intelligence - November 2016
Computational Intelligence - November 2016 - Cover1
Computational Intelligence - November 2016 - Cover2
Computational Intelligence - November 2016 - 1
Computational Intelligence - November 2016 - 2
Computational Intelligence - November 2016 - 3
Computational Intelligence - November 2016 - 4
Computational Intelligence - November 2016 - 5
Computational Intelligence - November 2016 - 6
Computational Intelligence - November 2016 - 7
Computational Intelligence - November 2016 - 8
Computational Intelligence - November 2016 - 9
Computational Intelligence - November 2016 - 10
Computational Intelligence - November 2016 - 11
Computational Intelligence - November 2016 - 12
Computational Intelligence - November 2016 - 13
Computational Intelligence - November 2016 - 14
Computational Intelligence - November 2016 - 15
Computational Intelligence - November 2016 - 16
Computational Intelligence - November 2016 - 17
Computational Intelligence - November 2016 - 18
Computational Intelligence - November 2016 - 19
Computational Intelligence - November 2016 - 20
Computational Intelligence - November 2016 - 21
Computational Intelligence - November 2016 - 22
Computational Intelligence - November 2016 - 23
Computational Intelligence - November 2016 - 24
Computational Intelligence - November 2016 - 25
Computational Intelligence - November 2016 - 26
Computational Intelligence - November 2016 - 27
Computational Intelligence - November 2016 - 28
Computational Intelligence - November 2016 - 29
Computational Intelligence - November 2016 - 30
Computational Intelligence - November 2016 - 31
Computational Intelligence - November 2016 - 32
Computational Intelligence - November 2016 - 33
Computational Intelligence - November 2016 - 34
Computational Intelligence - November 2016 - 35
Computational Intelligence - November 2016 - 36
Computational Intelligence - November 2016 - 37
Computational Intelligence - November 2016 - 38
Computational Intelligence - November 2016 - 39
Computational Intelligence - November 2016 - 40
Computational Intelligence - November 2016 - 41
Computational Intelligence - November 2016 - 42
Computational Intelligence - November 2016 - 43
Computational Intelligence - November 2016 - 44
Computational Intelligence - November 2016 - 45
Computational Intelligence - November 2016 - 46
Computational Intelligence - November 2016 - 47
Computational Intelligence - November 2016 - 48
Computational Intelligence - November 2016 - 49
Computational Intelligence - November 2016 - 50
Computational Intelligence - November 2016 - 51
Computational Intelligence - November 2016 - 52
Computational Intelligence - November 2016 - 53
Computational Intelligence - November 2016 - 54
Computational Intelligence - November 2016 - 55
Computational Intelligence - November 2016 - 56
Computational Intelligence - November 2016 - 57
Computational Intelligence - November 2016 - 58
Computational Intelligence - November 2016 - 59
Computational Intelligence - November 2016 - 60
Computational Intelligence - November 2016 - 61
Computational Intelligence - November 2016 - 62
Computational Intelligence - November 2016 - 63
Computational Intelligence - November 2016 - 64
Computational Intelligence - November 2016 - 65
Computational Intelligence - November 2016 - 66
Computational Intelligence - November 2016 - 67
Computational Intelligence - November 2016 - 68
Computational Intelligence - November 2016 - 69
Computational Intelligence - November 2016 - 70
Computational Intelligence - November 2016 - 71
Computational Intelligence - November 2016 - 72
Computational Intelligence - November 2016 - Cover3
Computational Intelligence - November 2016 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202311
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202308
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202305
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202302
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202211
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202208
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202205
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202202
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202111
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202108
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202105
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202102
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202011
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202008
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202005
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_202002
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201911
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201908
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201905
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201902
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201811
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201808
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201805
https://www.nxtbook.com/nxtbooks/ieee/computationalintelligence_201802
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring17
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring16
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring15
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring14
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_summer13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_spring13
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_winter12
https://www.nxtbook.com/nxtbooks/ieee/computational_intelligence_fall12
https://www.nxtbookmedia.com