IEEE Signal Processing - May 2018 - 36

a Bit-Rate Constraint" section. In addition, steps 3 and 4
may be replaced by different techniques to attain the optimal lossy compression performance [6]. For example, the
output of each sampling branch can be encoded independently of the other outputs using a separate bitstream. The
bit rate of each bitstream is determined by the water-filling
principle of (6b), with the PSD replaced by the PSD of the
filtered signal at each sampling branch. Finally, we note that
the multibranch uniform sampler can be replaced by a nonuniform sampler with a single branch and possibly timevarying operation [54], or fewer uniform sampling branches
of different sampling rates. That is, although uniform multibranch sampling attains the minimal distortion D (fs, R) , it
may not achieve it using the most compact system implementation. In addition to these extensions, we note that the
characterization of the minimal distortion in ADX has also
been derived for the Wiener process and for sparse source
signals [62], [63].

Consider a Gaussian stationary process X X (t) with a PSD of
S X (f) =

1/f0
, f0 2 0.
(rf/f0) 2 + 1

(15)

The signal X X (t) is also a Markov process, and it is, in fact,
the unique Gaussian stationary process that is also Markovian
(also known as the Ornstein-Uhlenbeck process). The PSD
S X (f) is shown in Figure 15, along with the relation between
the bit rate R and the minimal sampling frequency f R required
to achieve Shannon's DRF of X X (t) . This relation is obtained
by evaluating D (fs, R) for the PSD S X (f) . In fact, the exact
equation describing the green curve in Figure 15 can be evaluated in closed form, from which it follows [18] that
arctan ^rfR /f0 h
m.
R = 1 c fR - f0
ln 2
r/2

(16)

Notice that, although the Nyquist frequency of the signal
in this example is infinite, for any finite R, there exists a critical sampling frequency f R, satisfying (16), such that Shannon's
Applications
DRF
of X X (t) can be attained by sampling at or above f R.
The most straightforward application of sampling according to
the optimal ADX scheme is the possibility to reduce the samThe asymptotic behavior of (16) as R extends to infinity is
pling rates in systems operating under bit-rate restrictions.
given by R~ ^ fR / ln 2h . Thus, for R sufficiently large, the optiExamples are listed in "System Constraints
mal sampling rate is linearly proportional to
on Bit Rate." These systems process inforR and, in particular, in the limit of zero disThe lack of computational tortion when R grows to infinity. The ratio
mation that originated in an analog signal
resources for the
under a bit-rate constraint. Therefore, in
R/fs is the average number of bits per sample
extraction of useful
these cases, the rate at which the analog
used in the resulting digital representation.
input is sampled can be reduced to be as
It follows from (16) that, asymptotically, the
information from large
low as the critical sampling rate f R, without
right number of bits per sample converges to
data sets is one of the
increasing the overall distortion. How low
1/ ln 2 . 1.45 . If the number of bits per sammost pressing issues of
this f R is, compared to the Nyquist rate
ple is below this value, then the distortion
the digital age.
in ADX is dominated by Shannon's DRF of
or the spectral occupancy of the signal,
depends on our assumptions on the source
X X (t) , as there are not enough bits to repstatistics through its PSD. Examples for the dependency
resent the information acquired by the sampler. If the number
between the two are shown in Figure 15. Evidently, reducing
of bits per sample is greater than this value, then the distortion
the sampling rate allows the saving of other system paramein ADX is dominated by the sampling distortion, as there are
ters, such as power and thermal noise resulting from lower
not enough samples for describing the signal up to a distortion
clock cycles. Alternatively, this reduction provides a way to
equals to its Shannon's DRF.
sample wide-band signals that cannot be sampled at their
As a numerical example, assume that we encode X X (t)
Nyquist rate without introducing additional distortion due to
using two bits per sample, i.e., fs = 2R . As R " 3, the ratio
sampling, on top of the distortion due to a bit-rate constraint.
between the minimal distortion D (fs, R) and Shannon's DRF
Next, we explore additional theoretical and practical implicaof the signal converges to approximately 1.08, whereas the
tions of our ADX scheme.
ratio between D (fs, R) and mmse ( fs) converges to approximately 1.48. In other words, it is possible to attain the optimal
encoding performance within a gap of approximately 8% by
Sampling infinite bandwidth signals
providing one sample per each two bits per unit time used in
While a common assumption in signal processing is that
this encoding. Alternatively, it is possible to attain the optimal
for all practical purposes the bandwidth of the source sigsampling performance within a gap of approximately 48% by
nal is bounded, there are many important cases where this
providing two bits per each sample taken.
assumption does not hold. These cases include Markov
processes, autoregressive processes, and the Wiener process or other semimartingales. An important contribution
Theoretical limits on estimation from
of the ADX paradigm is in describing the optimal tradeoff
sampled and quantized information
among distortion, sampling rate, and bit rate, even if the
The limitation on bit rate in the scenarios mentioned in
source signal is bandlimited. This tradeoff is best explained
"System Constraints on Bit Rate" are the result of engiby an example.
neering limitations. However, sampling and quantization

IEEE Signal Processing Magazine

May 2018

Table of Contents for the Digital Edition of IEEE Signal Processing - May 2018