IEEE Signal Processing - May 2018 - 19

bit-rate constraint, since it describes the minimal sampling rate
required to attain the optimal performance in systems operating under quantization or bit-rate restrictions. Therefore, the
critical sampling rate extends the minimal-distortion sampling
rate considered by Shannon, Nyquist, and Landau. It is only as
the bit rate extends to infinity that sampling at the Nyquist rate
is necessary to attain minimal (i.e., zero) distortion for general
input distributions.
Figure 2 represents a general block diagram for systems
that process information through sampling and are limited
in the number of bits they can transmit per unit time, the
amount of memory they use, or the number of states they can
assume. Therefore, the critical sampling rate that arises in
this setting describes the fundamental limit of sampling in
systems like audio and video recorders, radio receivers, and
digital cameras. Moreover, this model also includes signal
processing techniques that use sampling and operate under
bit-rate constraints, such as artificial neural networks [8],
financial markets analyzers [9], and techniques to accelerate operations over large data sets by sampling [10]. In "System Constraints on Bit Rate," we list a few scenarios where
sampling and bit-rate restrictions arise in practice. Other

utilizations of the ADX paradigm will be discussed in the
"Applications" section.
To derive the critical sampling rate, we rely on the following two steps:
1) Given the output of the sampler, derive the optimal way to
encode these samples subject to the bit rate R so as to minimize the MSE distortion in reconstructing the original continuous-time signal.
2) Derive the optimal sampling scheme that minimizes the
MSE in the first step subject to the sampling rate constraint.
When the analog signal can be perfectly recovered from
the output of the sampler, the fundamental distortion limit
in step 1 depends only on the bit-rate constraint and leads to
Shannon's DRF. In this article, we explore this function as well
as the optimal encoding to attain it. Applications of the ADX
framework and the critical sampling rate that attains the minimal distortion are also discussed.
Before exploring the minimal distortion limit in the ADX
setting, it is instructive to consider the distortion in pulse-code
modulation, which is a particular system that is implementing a simple version of a sampler, an encoder, and a decoder.
Although this system does not implement the optimal sampling

System Constraints on Bit Rate
The analog-to-digital compression setting of Figure 2 is relevant to any system that processes information by sampling and is subject to a bit-rate constraint. Three possible
restrictions on a system's bit rate that arise in practice are
as follows:
* Memory: Digital systems often operate under a constraint on the amount of memory or the states they can
assume. Under such a restriction, the bit rate is the normalized amount of memory used over time (or the
dimension of the source signal). For example, consider
a system of K states that analyzes information obtained
by observing an analog signal for T seconds. The maximal bit rate of the system is R = log 2 (K) /T.
* Power: Emerging sensor network technologies, such as
those developed for biomedical applications and smart
cities, use many low-cost sensors to collect and transmit
data to remote locations [11]. These sensors must operate
under severe power restrictions and, hence, are limited
by the number of comparisons in their analog-to-digital
conversion (ADC) operation. These comparisons are typically the most energy-consuming part of the ADC unit, so
that the total power consumption in an ADC unit is proportional to the number of comparisons [12, Sec. 2.1].
In general, the number of comparisons is proportional to
the bit rate, since any output of bit rate R is generated by
at least R comparisons (although the exact number
depends on the particular implementation of the ADC
and may even grow exponentially in the bit rate [13]).

Therefore, power restrictions lead to a bit-rate constraint
and to an MSE distortion floor given by Shannon's distortion-rate function of the analog input signal.
An important scenario of power-restricted ADC units
arises in wireless communication using millimeter waves
[14]. Severe path loss of electromagnetic waves in
these frequencies is compensated for by using a large
number of receiver antennas. Each antenna is associated with a radio-frequency chain that includes an ADC
unit. Because of the resulting large number of ADC
units, power consumption is one of the major engineering challenges in millimeter-wave communication.
* Communication: Low-power sensors may also be limited by the rates of communication available to send
their digital sensed information to a remote location.
For example, consider a low-energy device collecting
medical signals and transmitting its measurements
wirelessly to a central processor (e.g., a smartphone).
The communication rate from the sensor to the central
processor depends on the capacity of the channel
between them, which is a function of the available
transmit power for communication. When the transmit
power is limited, so is the capacity. As a result, the
data rate associated with the digital representation of
the sensed information cannot exceed this capacity
limit, since, without additional processing, there is no
point in collecting more information than what can
be communicated.

IEEE Signal Processing Magazine

May 2018

Table of Contents for the Digital Edition of IEEE Signal Processing - May 2018