IEEE Signal Processing - May 2018 - 18

Sampling

Quantization

0100101001101010110101
1101001101001110111111
0010001011110010001010

FIGURE 1. An illustration showing that analog-to-digital conversion (ADC) is achieved by combining sampling and quantization. (Image of the guitar
courtesy of https://pixabay.com; image of Van Gogh's The Starry Night courtesy of Wikipedia.)

unit time of the continuous-time source signal, and the bit rate
R is the number of bits per unit time used in the representation
of these samples. The resulting system describing our problem
is shown in Figure 2 and is referred to as the analog-to-digital
compression (ADX) setting.
The digital representation in this setting is obtained by
transforming a continuous-time, continuous-amplitude, random source signal X (t) through a concatenated operation
of a sampler and an encoder, resulting in a bit sequence. For
instance, when the input signal X (t) is observed over a time
interval T, then the sampler produces 6 fs T @ samples, and the
encoder maps these samples to 6TR@ bits. The decoder estimates the original analog signal from this bit sequence. The distortion is defined to be the mean squared error (MSE) between
the input signal X (t) and its reconstruction Xt (t). Since we
are interested in the fundamental distortion limit subject to a
sampling constraint, we allow optimization over the encoder,
decoder, and time horizon T. In addition, we also explore the
optimal sampling mechanism but limit ourselves to the class of
linear and continuous deterministic samplers [7]. That is, each
sampler in this class is a linear continuous mapping of signals
over time lag T to R6 fs T @.

Sampler

Encoder

R bits
s

Distortion

Unlimited Sampling Rate
(Shannon's DRF)

X (t )

fs samples
s

The minimal distortion in ADX is bounded from below
by two extreme cases of the sampling rate and the bit rate, as
shown in Figure 3:
1) When the bit rate R is unlimited, the minimal ADX distortion reduces to the MSE in interpolating a signal from its
samples at rate fs .
2) When the sampling rate fs is unlimited or above the
Nyquist rate of the signal, the ADX distortion reduces to
Shannon's DRF of the signal.
Indeed, in this situation, the optimal encoder can recover
the original continuous-time signal without distortion and
then encode this recovery in an optimal manner according to
the scheme that attains Shannon's DRF. Our goal is therefore
to characterize the MSE due to the joint effect of a finite bitrate constraint and sampling at a sub-Nyquist sampling rate
or for signals that are not bandlimited. In particular, we are
interested in the minimal sampling rate for which Shannon's
DRF, describing the minimal distortion subject to a bit-rate
constraint, is attained. As shown in Figure 3 and as will be
explained in more detail in this article, this sampling rate is
usually below the Nyquist rate of the signal. We denote this
minimal sampling rate as the critical sampling rate subject to a

Distortion
X (t )

Analog
(Continuous Time)

Decoder

Analog
(Discrete Time)

Unlimited
Bit Rate
?

fNyq

Digital

FIGURE 2. The ADX and reconstruction setting. Our goal is to derive the
minimal distortion between the signal and its reconstruction from any encoding, at bit rate R of the samples of the signal taken at sampling rate fs .

Distortion in ADX = ?

FIGURE 3. The minimal sampling rate for attaining the minimal distortion
achievable in the presence of quantization is usually below the Nyquist
rate, whereas sampling at the Nyquist rate is necessary to attain zero
distortion without quantization constraints.

IEEE Signal Processing Magazine

May 2018

https://www.pixabay.com

Table of Contents for the Digital Edition of IEEE Signal Processing - May 2018