Computational Intelligence - November 2016 - 21

MMSE

3. performance Comparison
From a Bayesian perspective it can be interpreted
In Figure 2, the average reconstruction error
(NMSE) with increasing support cardinality
that for each algorithm a fixed prior distribution has
is plotted for the case where the non-zero
been employed. But in real life that assumed prior
coefficients are from a zero mean, unit varidistribution may be significantly different from the
ance, Gaussian distribution. It is evident that
our proposed adaptive approach gives the
true distribution of the data.
best performance in this case with Reweighted , 2 minimization approach being a very
distributional parameters our unified MAP estimation
close competitor. It is worth noting that Reweighted , 2
framework leads to several popular SSR algorithms, specifiworks much better compared to the other two methods
cally reweighted algorithms available in the literature. Based
with fixed distributional parameters, and the reason being
on this result, we proposed an adaptive framework where
the heuristic update of e, which helps it to get away from
the distributional parameters of GT have not been fixed
local minima. Hence, e update in Reweighted , 2 is absobeforehand, and adapted based on the measurement over
lutely necessary as we have found out for fixed e this algoiterations. Our extensive empirical results also show the effirithm's performance decreases significantly. This heuristic
cacy of this adaptive approach over other MAP estimation
update of e falls into the adaptive paradigm but our probased SSR algorithms.
posed approach provides a systematic approach to adapt
both the distributional parameters.
In Figure 3, the average reconstruction error (NMSE)
with increasing support cardinality is plotted where the
non-zero coefficients are generated from a Laplace distri0.35
bution with variance 1. Again, the empirical superiority of
0.30
our proposed adaptive approach over other algorithms is
0.25
evident from Figure 3. An interesting point to note here, is
0.20
that Reweighted , 1 with fixed e matches the performance
of Reweighted , 2 with e update and the reason could be
0.15
that true distribution of the non-zero coefficients and
0.10
the assumed prior for Reweighted , 1 have the similar
0.05
tail behavior.
25
30
35
40
45
50
Finally, we repeat the same set of experiments where the
k
non-zero coefficients follow a sub Gaussian distribution, i.e.
Reweighted /2
Adaptive /p
Uniform ±1 random spikes, and the plot of the average reconReweighted /1
LASSO (/1)
struction error (NMSE) with increasing support cardinality is
shown in Figure 4. The performance improvement of the proFigure 2 recovery performance with Gaussian distributed non-zero
posed approach over other reweighted algorithms is significant
coefficients.
compared to the previous two cases.
In Figure 5 we visualize the adaptation of the distributional
parameters step. We consider a case where k = 30 and the nonzero entries have been randomly drawn from zero mean and
0.24
unit variance Gaussian distribution. On top, the absolute value
0.22
of the true non-zero coefficients have been plotted. Next two
0.20
figures represent the learned values of q and p after learning
0.18
them using our proposed algorithm. We see how the adaptation
0.16
0.14
step helps us to learn the corresponding distributional parame0.12
ters which plays a key role in modeling the tail nature of the
0.10
sparsity inducing prior distributions.
0.08
0.06

V. Conclusion and Discussion

In this paper, we formulated the SSR problem from a Bayesian perspective and discussed a generalized scale mixture
family: PESM in detail. We analyzed the tail behavior of the
GT class of distributions, which is a member of PESM family, and discussed when a GT distribution will be suitable to
model sparsity. We also showed, how by choosing specific

k
Reweighted /2
Reweighted /1

Adaptive /p
LASSO (/1)

Figure 3 recovery performance with super Gaussian (laplace)
distributed non-zero coefficients.

novEmbEr 2016 | IEEE ComputatIonal IntEllIgEnCE magazInE

Table of Contents for the Digital Edition of Computational Intelligence - November 2016