Signal Processing - January 2016 - 127

which ensures that m (n) attains values 1 and 0 for a (n) = a + and
a (n) = - a +, respectively. Using (18), the cvx-PN-LMS rule
becomes
a (n + 1) = a (n) +

na

p (n)

e (n) [y 1 (n) - y 2 (n)]

sgm [a (n)] " 1 - sgm [a (n)] , | -a a + .
+

(19)

As we can see, a (n) is truncated at every iteration to keep it inside
the interval [- a +, a +] .
When deciding between affine or convex combinations with
the above gradient-based rules, one should be aware of the following facts:
■ Optimally adjusted affine combinations attain smaller EMSE
than convex ones in some situations (cases 1 and 2 of Table 1).
■ Due to the constraints on the value of m (n), cvx-LMS and
cvx-PN-LMS do not diverge, and large values of the step size
n a can be used.
■ Gradient adaptation of the combination parameter implies
that combinations introduce some additional gradient noise,
which should be minimized with an adequate selection of n a
or n m . In this sense, the factor m (n) [1 - m (n)] that appears
in the adaptation of a (n) with cvx-PN-LMS reduces the gradient noise when the mixing parameter becomes close to 0 or 1.
In our experience, the last two issues have an important impact
on the performance of the combination, to the extent that convex combinations may actually be preferred over affine ones,
unless in situations where significant EMSE gains can be anticipated for the affine combination from the theoretical analysis.
The next section will compare the performance of these rules
through simulations.
We should point out that other rules can be found in the literature for the adaptation of the mixing parameter, such as the least
squares rule from [39], the sign algorithm proposed in [41], or a
method relying on the ratio between the estimated errors of the
filter components [16]. Nevertheless, in the following we will
restrict our discussion to the methods that have been presented in
this section, which are the most frequently used in the literature.
ConvErgEnCE propErtiEs of CoMbination fiLtErs
To examine the convergence properties of aff-PN-LMS and cvx-PNLMS, we consider the combination of two normalized LMS (NLMS)
filters with step sizes n 1 = 0.5 and n 2 = 0.01. The optimum solution is a stationary vector of length seven, the covariance matrix of
the input signal is R = (1/7) I, and the variance of the observation
noise is adjusted to get an SNR of 20 dB. Different step sizes have
been explored for the combination: n a = [0.25, 0.5, 1] for cvx-PNLMS, while n m = n a /800 is used for aff-PN-LMS to get comparable
steady-state error. Regarding these step-size values, we can see that
the range of practical values for n a is within the usual range of steps
sizes used with normalized schemes, whereas for the affine combination much smaller values are required for comparable performance.
This fact simplifies the selection of the step size in the convex case.
Figure 4 illustrates the performance of affine and convex combinations averaged over 1,000 experiments. Figure 4(b)-(d)

compares the convergence of both schemes with respect to the
optimum selection of the mixing parameter given by (8). In all
cases, the combination schemes converge first to the EMSE level
of the fast filter (- 30 dB), and after a while follow the slow component to get a final EMSE of around - 50 dB. It is interesting to
see that cvx-PN-LMS shows near optimum selection of the mixing
parameter for all three values of n a, while the affine combination
may incur a significant delay, especially for the smallest n m .
Figure 4(a) plots the excess steady-state error of both schemes
with respect to g 2 (3) as a function of the step size, and shows
that this faster convergence of cvx-PN-LMS with respect to aff-PNLMS is not in exchange of larger residual error.
The fact that cvx-PN-LMS has the ability to switch rapidly
between the fast and slow filter components while at the same
time minimizing the residual error in steady state is due to the
incorporation of the activation function, whose derivative propagates to the update rule. In other words, we can view the effective
step size of cvx-PN-LMS as being n a m (n) [1 - m (n)] (see Table 4),
and the evolution of the multiplicative factor, represented in
Figure 4(e), shows that this effective step size becomes large when
the combination needs to switch between filter components, while
becoming small in steady state, thus minimizing the residual
error after convergence is complete.
bEnEfits of powEr norMaLizEd updating ruLEs
To illustrate the benefits of power normalized updating rules, we
compare the behavior of a convex combination of two NLMS filters with step sizes n 1 = 0.5 and n 2 = 0.01, employing both the
cvx-LMS rule and its power normalized version (cvx-PN-LMS) for
the combination layer. The optimum solution is a length-30 nonstationary vector, which varies according to the random-walk
model given by (11). The covariance matrices of the change of the
optimum solution and of the input signal are given respectively by
Q = v 2q I and R = (1/30) I. The variance of the observation noise
is adjusted to get different SNR levels. The step size for the cvxLMS rule has been set to n a = 1, 000 and for its power normalized version (cvx-PN-LMS), we have set n a = 1.
Figure 5 shows the steady-state NSD of the individual filters
and of their convex combinations obtained with the cvx-LMS rule
and with the cvx-PN-LMS scheme, as a function of the speed of
changes of the optimum solution [Tr (Q) = 30v 2q] . The left panel
considers an SNR of 5 dB and the right panel, an SNR of 30 dB.
We can observe that the combination scheme with the cvx-LMS
rule results in a suboptimal performance when the optimum solution changes very fast [for large Tr (Q)] . This is due to the fact
that both component filters are incurring a very significant error,
resulting in a nonnegligible gradient noise when updating the
auxiliary parameter a (n) with cvx-LMS [37]. The performance of
cvx-LMS also degrades, whatever the value of Tr (Q), as the SNR
decreases. On the other hand, when the cvx-PN-LMS rule is
employed, the combination shows a very stable operation, and
behaves as well as the best component filter not only for any SNR,
but also for all values of Tr (Q) . A similar behavior is also observed
when we compare the aff-LMS rule to its power normalized version (aff-PN-LMS) to update m (n) in affine combinations [18].

IEEE SIGNAL PROCESSING MAGAZINE [127] jANuARy 2016



Table of Contents for the Digital Edition of Signal Processing - January 2016

Signal Processing - January 2016 - Cover1
Signal Processing - January 2016 - Cover2
Signal Processing - January 2016 - 1
Signal Processing - January 2016 - 2
Signal Processing - January 2016 - 3
Signal Processing - January 2016 - 4
Signal Processing - January 2016 - 5
Signal Processing - January 2016 - 6
Signal Processing - January 2016 - 7
Signal Processing - January 2016 - 8
Signal Processing - January 2016 - 9
Signal Processing - January 2016 - 10
Signal Processing - January 2016 - 11
Signal Processing - January 2016 - 12
Signal Processing - January 2016 - 13
Signal Processing - January 2016 - 14
Signal Processing - January 2016 - 15
Signal Processing - January 2016 - 16
Signal Processing - January 2016 - 17
Signal Processing - January 2016 - 18
Signal Processing - January 2016 - 19
Signal Processing - January 2016 - 20
Signal Processing - January 2016 - 21
Signal Processing - January 2016 - 22
Signal Processing - January 2016 - 23
Signal Processing - January 2016 - 24
Signal Processing - January 2016 - 25
Signal Processing - January 2016 - 26
Signal Processing - January 2016 - 27
Signal Processing - January 2016 - 28
Signal Processing - January 2016 - 29
Signal Processing - January 2016 - 30
Signal Processing - January 2016 - 31
Signal Processing - January 2016 - 32
Signal Processing - January 2016 - 33
Signal Processing - January 2016 - 34
Signal Processing - January 2016 - 35
Signal Processing - January 2016 - 36
Signal Processing - January 2016 - 37
Signal Processing - January 2016 - 38
Signal Processing - January 2016 - 39
Signal Processing - January 2016 - 40
Signal Processing - January 2016 - 41
Signal Processing - January 2016 - 42
Signal Processing - January 2016 - 43
Signal Processing - January 2016 - 44
Signal Processing - January 2016 - 45
Signal Processing - January 2016 - 46
Signal Processing - January 2016 - 47
Signal Processing - January 2016 - 48
Signal Processing - January 2016 - 49
Signal Processing - January 2016 - 50
Signal Processing - January 2016 - 51
Signal Processing - January 2016 - 52
Signal Processing - January 2016 - 53
Signal Processing - January 2016 - 54
Signal Processing - January 2016 - 55
Signal Processing - January 2016 - 56
Signal Processing - January 2016 - 57
Signal Processing - January 2016 - 58
Signal Processing - January 2016 - 59
Signal Processing - January 2016 - 60
Signal Processing - January 2016 - 61
Signal Processing - January 2016 - 62
Signal Processing - January 2016 - 63
Signal Processing - January 2016 - 64
Signal Processing - January 2016 - 65
Signal Processing - January 2016 - 66
Signal Processing - January 2016 - 67
Signal Processing - January 2016 - 68
Signal Processing - January 2016 - 69
Signal Processing - January 2016 - 70
Signal Processing - January 2016 - 71
Signal Processing - January 2016 - 72
Signal Processing - January 2016 - 73
Signal Processing - January 2016 - 74
Signal Processing - January 2016 - 75
Signal Processing - January 2016 - 76
Signal Processing - January 2016 - 77
Signal Processing - January 2016 - 78
Signal Processing - January 2016 - 79
Signal Processing - January 2016 - 80
Signal Processing - January 2016 - 81
Signal Processing - January 2016 - 82
Signal Processing - January 2016 - 83
Signal Processing - January 2016 - 84
Signal Processing - January 2016 - 85
Signal Processing - January 2016 - 86
Signal Processing - January 2016 - 87
Signal Processing - January 2016 - 88
Signal Processing - January 2016 - 89
Signal Processing - January 2016 - 90
Signal Processing - January 2016 - 91
Signal Processing - January 2016 - 92
Signal Processing - January 2016 - 93
Signal Processing - January 2016 - 94
Signal Processing - January 2016 - 95
Signal Processing - January 2016 - 96
Signal Processing - January 2016 - 97
Signal Processing - January 2016 - 98
Signal Processing - January 2016 - 99
Signal Processing - January 2016 - 100
Signal Processing - January 2016 - 101
Signal Processing - January 2016 - 102
Signal Processing - January 2016 - 103
Signal Processing - January 2016 - 104
Signal Processing - January 2016 - 105
Signal Processing - January 2016 - 106
Signal Processing - January 2016 - 107
Signal Processing - January 2016 - 108
Signal Processing - January 2016 - 109
Signal Processing - January 2016 - 110
Signal Processing - January 2016 - 111
Signal Processing - January 2016 - 112
Signal Processing - January 2016 - 113
Signal Processing - January 2016 - 114
Signal Processing - January 2016 - 115
Signal Processing - January 2016 - 116
Signal Processing - January 2016 - 117
Signal Processing - January 2016 - 118
Signal Processing - January 2016 - 119
Signal Processing - January 2016 - 120
Signal Processing - January 2016 - 121
Signal Processing - January 2016 - 122
Signal Processing - January 2016 - 123
Signal Processing - January 2016 - 124
Signal Processing - January 2016 - 125
Signal Processing - January 2016 - 126
Signal Processing - January 2016 - 127
Signal Processing - January 2016 - 128
Signal Processing - January 2016 - 129
Signal Processing - January 2016 - 130
Signal Processing - January 2016 - 131
Signal Processing - January 2016 - 132
Signal Processing - January 2016 - 133
Signal Processing - January 2016 - 134
Signal Processing - January 2016 - 135
Signal Processing - January 2016 - 136
Signal Processing - January 2016 - 137
Signal Processing - January 2016 - 138
Signal Processing - January 2016 - 139
Signal Processing - January 2016 - 140
Signal Processing - January 2016 - 141
Signal Processing - January 2016 - 142
Signal Processing - January 2016 - 143
Signal Processing - January 2016 - 144
Signal Processing - January 2016 - 145
Signal Processing - January 2016 - 146
Signal Processing - January 2016 - 147
Signal Processing - January 2016 - 148
Signal Processing - January 2016 - 149
Signal Processing - January 2016 - 150
Signal Processing - January 2016 - 151
Signal Processing - January 2016 - 152
Signal Processing - January 2016 - 153
Signal Processing - January 2016 - 154
Signal Processing - January 2016 - 155
Signal Processing - January 2016 - 156
Signal Processing - January 2016 - 157
Signal Processing - January 2016 - 158
Signal Processing - January 2016 - 159
Signal Processing - January 2016 - 160
Signal Processing - January 2016 - 161
Signal Processing - January 2016 - 162
Signal Processing - January 2016 - 163
Signal Processing - January 2016 - 164
Signal Processing - January 2016 - 165
Signal Processing - January 2016 - 166
Signal Processing - January 2016 - 167
Signal Processing - January 2016 - 168
Signal Processing - January 2016 - Cover3
Signal Processing - January 2016 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201809
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201807
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201805
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201803
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201801
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1117
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0917
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0717
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0517
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0317
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0117
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1116
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0916
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0716
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0516
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0316
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0116
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1115
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0915
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0715
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0515
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0315
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0115
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1114
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0914
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0714
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0514
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0314
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0114
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1113
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0913
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0713
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0513
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0313
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0113
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1112
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0912
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0712
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0512
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0312
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0112
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1111
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0911
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0711
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0511
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0311
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0111
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1110
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0910
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0710
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0510
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0310
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0110
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1109
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0909
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0709
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0509
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0309
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0109
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1108
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0908
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0708
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0508
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0308
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0108
https://www.nxtbookmedia.com