IEEE Signal Processing - May 2018 - 17

©iStockphoto.com/agSandrew

focuses only on the critical sampling rate required to perfectly reconstruct a bandlimited signal from its discrete samples.
It does not incorporate the quantization precision of the samples and does not apply to signals that are not bandlimited. It
is, in fact, impossible to obtain an exact representation of any
continuous-amplitude sequence of samples by a digital
sequence of numbers because of finite quantization precision, and, therefore, any digital representation of an analog
signal is prone to error. That is, no continuous-amplitude signal can be reconstructed from its quantized samples with
zero distortion regardless of the sampling rate, even when the
signal is bandlimited.
This limitation raises the following question: In converting
a signal to bits via sampling and quantization at a given bit
precision, can the signal be reconstructed from these samples

with minimal distortion based on sub-Nyquist sampling? In
this article, we discuss this question by extending classical
sampling theory to account for quantization and for nonbandlimited inputs. That is, for an arbitrary stochastic input and
given a total budget of quantization bits, we consider the lowest
sampling rate required to sample the signal such that reconstruction of the signal from its quantized samples results in
minimal distortion. Without assuming any particular structure
of the input analog signal, this sampling rate is often below the
signal's Nyquist rate.
The minimal distortion achievable in the presence of quantization depends on the particular way the signal is quantized
or, more generally, encoded into a sequence of bits. Since we
are interested in the fundamental distortion limit in recovering an analog signal from its digital representation, we consider all possible encoding and reconstruction (decoding)
techniques. As an example, in Figure 1, the smartphone display may be viewed as a reconstruction of the real-world painting The Starry Night from its digital representation. No matter
how excellent the quality of a smartphone's high-definition
screen may be, this recovery is not perfect, since the digital
representation of the analog image is not accurate due to a loss
of information occurring during the conversion from analog
to bits. Our goal is to analyze this loss as a function of hardware limitations on the sampling mechanism and the number
of bits used in the encoding. It is convenient to normalize this
number of bits by the signal's free dimensions, i.e., the dimensions along which new information is generated. For example,
the free dimensions of a visual signal are usually the horizontal and vertical axes of the frame, and the free dimension
of an audio wave is time. For simplicity, we consider analog
signals with a single free dimension, i.e., time. Therefore, our
restriction on the digital representation is given in terms of its
bit rate-the number of bits per unit time.
For an arbitrary continuous-time random signal with known
statistics, the fundamental distortion limit due to the encoding
of the signal using a limited bit rate is given by Shannon's distortion-rate function (DRF) [4]-[6]. This function provides the
optimal tradeoff between the bit rate of the signal's digital representation and the distortion in recovering the original signal
from this representation. Shannon's DRF is described only in
terms of the distortion criterion, the probability distribution on
the continuous-time signal, and the maximal bit rate allowed in
the digital representation. Consequently, the optimal encoding
scheme that attains Shannon's DRF is a general mapping from
continuous-time signal space to bits that does not consider
practical constraints in its implementation. In practice, the
encoding of an analog signal into bits entails first sampling the
signal and then representing the samples using a limited number of bits. Therefore, in practice, the minimal distortion in
recovering analog signals from their bit representation considers the digital encoding of the signal samples, with a constraint
on both the sampling rate and the bit rate of the system. Here,
the sampling rate fs is defined as the number of samples per

IEEE Signal Processing Magazine

May 2018

http://www.iStockphoto.com/agSandrew

Table of Contents for the Digital Edition of IEEE Signal Processing - May 2018