IEEE Signal Processing - May 2018 - 23

when the only restriction is the bit rate R of the resulting digital representation. In other words, we consider the minimal
distortion assuming that the encoder operates directly on the
continuous-time process X (t) , as shown in Figure 7.
This encoder observes a realization x (t) of the process X (t)
over some finite time horizon T and then represents its observation using 6TR@ bits. The number of possible states this encoding can take is, therefore, 26TR@ . As shown in Figure 8, without

0

DPCM (fs, R) (dB)

SΠ (f )

−5

−10

0

1
fs/fNyq

fixed bit rate R and the PSDs in the small frames. With a nonuniform energy
distribution, the optimal sampling rate of PCM is below the Nyquist rate.

Distortion
X (t )

Decoder
Analog

FIGURE 7. Encoding with full continuous-time source signal information.

"

x 0(t )

0...00

Encoder

x 1(t )

"

T
2

Digital

0...01
.
.
.

.
.
.
"

− T
2

R bits
s

Encoder

X (t )

TR bits
X (t )

1.5

FIGURE 6. The distortion in PCM as a function of the sampling rate fs for a

Minimal distortion subject to a bit-rate constraint
We now go back to the ADX setting of Figure 2. In this section, we consider the minimal distortion that can be attained

SΛ (f )

"

rate, since the covariance function of a bandlimited signal is continuous [23], [24]. This correlation is not exploited by the quantizer,
which maps two similar samples to the same digital value, leading to a redundant digital representation of the analog signal. Since
the overall bit rate is limited, this redundancy in representation is
translated to a higher distortion compared to the distortion in a lessredundant representation obtained at a lower sampling rate. In fact,
it is well known that the sampling rate that minimizes the distortion
in PCM also maximizes the entropy rate of the process postquantization, i.e., of YQ [n] [17]. Therefore, we conclude that the most
efficient representation of the analog signal in PCM under a bit-rate
constraint is attained by sampling at or below the Nyquist rate.
The previously discussed conclusions imply that we can readily improve the performance of PCM by providing a more compact representation of the signal in terms of bit rate under the same
distortion level, and we can do so in one of the following ways:
1) reduce the correlation between consecutive quantizer outputs by using a whitening transformation as in transform
coding [17] or by a delta feedback loop as in sigma-delta
modulation [25], [26]
2) compress the digital process YQ [n] using a universal lossless compressor, such as Lempel-Ziv [27], [28] or contexttree weighting [29]
3) aggregate a large block of, e.g., N samples of Y [n] and
r
represent these samples using a single index out of 2 RN possible values.
This last technique, commonly known as vector quantization [17], does not assume any restrictions on the mapping
from the samples to the digital representation, except the size
of the block. It therefore covers a wide range of quantization
techniques operating at bit rate R and includes 1) and 2) as
special cases. This technique leads to the most general way to
encode any discrete-time process to a digital representation,
subject only to a bit-rate constraint. Moreover, combined with
an optimal mechanism to represent the analog signal as a bit
sequence, this encoding technique attains the minimal distortion in encoding X (t) , described by Shannon's DRF D(R).

x 2TR - 1(t )

1...11

FIGURE 8. The optimal encoding with TR bits is obtained by mapping the source signal realization to the index of the predetermined reconstruction waveform
closest to this realization. The optimal set of reconstruction waveforms and the resulting average distortion are given by Shannon's source coding theorem.
IEEE Signal Processing Magazine

|

May 2018

|

23
Table of Contents for the Digital Edition of IEEE Signal Processing - May 2018
