IEEE Signal Processing - May 2018 - 25
The Water-Filling Scheme
In "Source Coding and the DRF," we explored the encoding of an independent and identically distributed (i.i.d.)
Gaussian sequence using a code of rate Rr bits per sample.
We now extend this source coding problem to consider the
joint encoding of m i.i.d. sequences taken from m Gaussian
distributions with variances v 21, f, v 2m, using a total of Rr
bits and under a sum mean squared error criterion.
From (S3) we see that it is possible to describe the ith
sequence using Rr i bits per symbol, such that / i R i # Rr
and the overall distortion with respect to all sequences is
m
D (R 1, f, R m) = / v i2 2 -2Ri.
The problem we consider next is how to allocate the total
bit budget Rr in a way that minimizes the overall distortion.
This is a convex problem whose solution can be expressed
by the following parametric expression [42, Ex. 5.2]:
R *i = 1 log +2 6v 2i /i@,
2
where i is chosen to satisfy the constraint R = / mi = 1 R *i .
The resulting distortion-rate function (DRF) is
D (Rr ) = D (R *1, f, R *m) = / min " v 2i , i ,.
m
i =1
This parametric expression for the DRF is referred to as a
water-filling scheme. The parameter i may be interpreted
as a water level, such that D (Rr ) is obtained by summing
the part of the variances that are below this level (see
Figure S1).
#-33 min " S X ( f ), i , df
Ri = 1
2
#-33 log + 6S X ( f ) /i@df,
θ
1
2
(6a)
(6b)
where [x] + is the maximum between x and zero. The parametric form of (6) has the graphical interpretation given by
Figure 9, denoted as the water-filling scheme. The distortion
in (6a) may be seen as if water is being poured into the area
bounded by the graph of S X ( f ) up to level i. The distortion
in (6a) is the total volume of the water. The bit rate is determined by integration over the preserved part through (6b). As
explained in "The Water-Filling Scheme," this approach is
obtained as the solution of an optimization problem involving
the allocation of the rate of the codes to describe different frequency components of the signal according to their respective
energy (components with higher energy are given a higher
code rate). As a result, in addition to the minimal distortion
subject only to the bit-rate constraint, the water-filling inter-
...
i
Preserved Spectrum
Lossy Compression Distortion
(S4)
i =1
D (R i) =
σi 2
FIGURE S1. The total distortion equals the sum of the part of the
variances that lie below the water-level i .
Intuitively, components with higher variance are described
with more bits, since they have a higher impact on the total
distortion. An interesting property of the water-filling scheme
is that, when R is small, the optimal coding does not allocate any bit budget to some of the components with the lowest variance. This means that no information is sent on these
low-variance components.
When the source is a stationary process, the DRF is
described by water-filling over the power spectral density
of the process, as in (6). In this case, different frequency
subbands correspond to different independent signal components, and (6) is obtained by solving an optimization
similar to that of minimizing (S4) over R 1, f, R m [36].
pretation provides the optimal coding scheme that attains this
minimal distortion [36]. Independent spectral components of
the signal are represented using independent bitstreams, where
the rate of each bitstream is determined according to the
water-filling principle.
The Pinsker-Kolmogorov expression (6) is easily adjusted to
account for a distortion criterion that assigns different weights
W ( f ) $ 0 to each spectral component. This spectral weighting is useful in applications where some tones are of different
importance than others, such as in psychoacoustic consideration in the digital encoding of audio signals [37]. The adjustment of the expression for the minimal distortion required
because of this importance weighting is achieved by evaluating the distortion equation (6a) with respect to W ( f ) S X ( f )
rather than S X ( f ), in a way similar to the procedure explained
in [38]. This different weighting emphasizes the generality of
the lossy compression principle. Under a strict bit-rate budget,
part of the analog signal must be removed due to lossy compression, and this part is the least important in our application.
IEEE Signal Processing Magazine
|
May 2018
|
25
Table of Contents for the Digital Edition of IEEE Signal Processing - May 2018
Contents
IEEE Signal Processing - May 2018 - Cover1
IEEE Signal Processing - May 2018 - Cover2
IEEE Signal Processing - May 2018 - Contents
IEEE Signal Processing - May 2018 - 2
IEEE Signal Processing - May 2018 - 3
IEEE Signal Processing - May 2018 - 4
IEEE Signal Processing - May 2018 - 5
IEEE Signal Processing - May 2018 - 6
IEEE Signal Processing - May 2018 - 7
IEEE Signal Processing - May 2018 - 8
IEEE Signal Processing - May 2018 - 9
IEEE Signal Processing - May 2018 - 10
IEEE Signal Processing - May 2018 - 11
IEEE Signal Processing - May 2018 - 12
IEEE Signal Processing - May 2018 - 13
IEEE Signal Processing - May 2018 - 14
IEEE Signal Processing - May 2018 - 15
IEEE Signal Processing - May 2018 - 16
IEEE Signal Processing - May 2018 - 17
IEEE Signal Processing - May 2018 - 18
IEEE Signal Processing - May 2018 - 19
IEEE Signal Processing - May 2018 - 20
IEEE Signal Processing - May 2018 - 21
IEEE Signal Processing - May 2018 - 22
IEEE Signal Processing - May 2018 - 23
IEEE Signal Processing - May 2018 - 24
IEEE Signal Processing - May 2018 - 25
IEEE Signal Processing - May 2018 - 26
IEEE Signal Processing - May 2018 - 27
IEEE Signal Processing - May 2018 - 28
IEEE Signal Processing - May 2018 - 29
IEEE Signal Processing - May 2018 - 30
IEEE Signal Processing - May 2018 - 31
IEEE Signal Processing - May 2018 - 32
IEEE Signal Processing - May 2018 - 33
IEEE Signal Processing - May 2018 - 34
IEEE Signal Processing - May 2018 - 35
IEEE Signal Processing - May 2018 - 36
IEEE Signal Processing - May 2018 - 37
IEEE Signal Processing - May 2018 - 38
IEEE Signal Processing - May 2018 - 39
IEEE Signal Processing - May 2018 - 40
IEEE Signal Processing - May 2018 - 41
IEEE Signal Processing - May 2018 - 42
IEEE Signal Processing - May 2018 - 43
IEEE Signal Processing - May 2018 - 44
IEEE Signal Processing - May 2018 - 45
IEEE Signal Processing - May 2018 - 46
IEEE Signal Processing - May 2018 - 47
IEEE Signal Processing - May 2018 - 48
IEEE Signal Processing - May 2018 - 49
IEEE Signal Processing - May 2018 - 50
IEEE Signal Processing - May 2018 - 51
IEEE Signal Processing - May 2018 - 52
IEEE Signal Processing - May 2018 - 53
IEEE Signal Processing - May 2018 - 54
IEEE Signal Processing - May 2018 - 55
IEEE Signal Processing - May 2018 - 56
IEEE Signal Processing - May 2018 - 57
IEEE Signal Processing - May 2018 - 58
IEEE Signal Processing - May 2018 - 59
IEEE Signal Processing - May 2018 - 60
IEEE Signal Processing - May 2018 - 61
IEEE Signal Processing - May 2018 - 62
IEEE Signal Processing - May 2018 - 63
IEEE Signal Processing - May 2018 - 64
IEEE Signal Processing - May 2018 - 65
IEEE Signal Processing - May 2018 - 66
IEEE Signal Processing - May 2018 - 67
IEEE Signal Processing - May 2018 - 68
IEEE Signal Processing - May 2018 - 69
IEEE Signal Processing - May 2018 - 70
IEEE Signal Processing - May 2018 - 71
IEEE Signal Processing - May 2018 - 72
IEEE Signal Processing - May 2018 - 73
IEEE Signal Processing - May 2018 - 74
IEEE Signal Processing - May 2018 - 75
IEEE Signal Processing - May 2018 - 76
IEEE Signal Processing - May 2018 - 77
IEEE Signal Processing - May 2018 - 78
IEEE Signal Processing - May 2018 - 79
IEEE Signal Processing - May 2018 - 80
IEEE Signal Processing - May 2018 - 81
IEEE Signal Processing - May 2018 - 82
IEEE Signal Processing - May 2018 - 83
IEEE Signal Processing - May 2018 - 84
IEEE Signal Processing - May 2018 - 85
IEEE Signal Processing - May 2018 - 86
IEEE Signal Processing - May 2018 - 87
IEEE Signal Processing - May 2018 - 88
IEEE Signal Processing - May 2018 - 89
IEEE Signal Processing - May 2018 - 90
IEEE Signal Processing - May 2018 - 91
IEEE Signal Processing - May 2018 - 92
IEEE Signal Processing - May 2018 - 93
IEEE Signal Processing - May 2018 - 94
IEEE Signal Processing - May 2018 - 95
IEEE Signal Processing - May 2018 - 96
IEEE Signal Processing - May 2018 - 97
IEEE Signal Processing - May 2018 - 98
IEEE Signal Processing - May 2018 - 99
IEEE Signal Processing - May 2018 - 100
IEEE Signal Processing - May 2018 - 101
IEEE Signal Processing - May 2018 - 102
IEEE Signal Processing - May 2018 - 103
IEEE Signal Processing - May 2018 - 104
IEEE Signal Processing - May 2018 - 105
IEEE Signal Processing - May 2018 - 106
IEEE Signal Processing - May 2018 - 107
IEEE Signal Processing - May 2018 - 108
IEEE Signal Processing - May 2018 - 109
IEEE Signal Processing - May 2018 - 110
IEEE Signal Processing - May 2018 - 111
IEEE Signal Processing - May 2018 - 112
IEEE Signal Processing - May 2018 - 113
IEEE Signal Processing - May 2018 - 114
IEEE Signal Processing - May 2018 - 115
IEEE Signal Processing - May 2018 - 116
IEEE Signal Processing - May 2018 - 117
IEEE Signal Processing - May 2018 - 118
IEEE Signal Processing - May 2018 - 119
IEEE Signal Processing - May 2018 - 120
IEEE Signal Processing - May 2018 - 121
IEEE Signal Processing - May 2018 - 122
IEEE Signal Processing - May 2018 - 123
IEEE Signal Processing - May 2018 - 124
IEEE Signal Processing - May 2018 - 125
IEEE Signal Processing - May 2018 - 126
IEEE Signal Processing - May 2018 - 127
IEEE Signal Processing - May 2018 - 128
IEEE Signal Processing - May 2018 - 129
IEEE Signal Processing - May 2018 - 130
IEEE Signal Processing - May 2018 - 131
IEEE Signal Processing - May 2018 - 132
IEEE Signal Processing - May 2018 - Cover3
IEEE Signal Processing - May 2018 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201809
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201807
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201805
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201803
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_201801
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1117
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0917
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0717
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0517
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0317
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0117
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1116
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0916
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0716
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0516
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0316
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0116
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1115
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0915
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0715
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0515
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0315
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0115
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1114
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0914
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0714
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0514
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0314
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0114
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1113
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0913
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0713
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0513
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0313
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0113
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1112
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0912
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0712
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0512
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0312
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0112
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1111
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0911
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0711
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0511
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0311
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0111
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1110
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0910
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0710
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0510
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0310
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0110
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1109
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0909
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0709
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0509
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0309
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0109
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_1108
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0908
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0708
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0508
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0308
https://www.nxtbook.com/nxtbooks/ieee/signalprocessing_0108
https://www.nxtbookmedia.com