IEEE Signal Processing - July 2018 - 13

that can change over time, albeit gradually. The problem of tracking such data (and
the subspaces) while being robust to outliers is called robust subspace tracking.
The third article in this four-part series
is by Pal and reviews new deterministic sampling techniques, reconstruction
algorithms, and performance guarantees
for low-rank correlation-driven estimation problems. In many problems, the
information of interest (often captured
in suitable parameters) is embedded in
the correlation of the data. In such cases,
it is possible to compress the raw data
and yet perfectly recover its correlation
matrix. Unlike conventional compressed
sensing, which uses a sparse representation of the data to achieve compression,
here compression can be achieved without sparsity, simply by exploiting certain
specific correlation structure of the data,
such as Toeplitz structure, that arise in a
large number of statistical signal processing and superresolution imaging problems. The article shows how to optimally
compress low-rank Toeplitz covariance
matrices using structured sampling ideas,
along with robust recovery algorithms
that do not require regularization. Fundamental lower bounds on source localization from such compressed covariance
matrices are also discussed. Finally, by
considering the sparse Bayesian learning
framework, it is illustrated that the ability
to exploit correlation structure in addition
to sparsity can enable significantly higher
levels of compression compared to what
can be attained by using sparsity alone.
All four articles focus on provable
solutions that are also fast and practically
useful, while also briefly reviewing everything else that exists. Detailed experimental comparisons help demonstrate the
practical implications of the theoretical
guarantees as well as make it easy for
a practitioner to pick the most suitable
approach. Detailed discussions of open
questions are also provided.
We end the series with the article by
Papyan et al. The great success of deep
learning in the past decade has been
mostly empirically based. Indeed, a solid
theory explaining the proposed architectures, the algorithms used, and the superb
performance obtained by this field has
been lagging behind. This article pres-

ents a systematic theoretical framework
for explaining deep learning based on
data modeling via sparse representation.
It proposes a multilayer sparse model
that describes the data's inner structure,
showing that decomposing these signals
into their building atoms amounts to various deep convolutional neural network
architectures. This observation is accompanied by a new and exciting ability to
theoretically analyze the performance of
these networks, posing clear conditions on
terms for their success. The article offers
a gradual and stand-alone description of
this story, starting from the general need
for models, passing through the story of
sparse representation theory and convolutional dictionaries, then turning to the
main message of tying these to the realm
of deep learning. Time will tell whether
this model-based view will be adopted
by our community as the gateway to the
much-needed theory for deep learning.

About the guest editor
Namrata Vaswani (namrata@iastate
.edu) received her B.Tech. degree from
the Indian Institute of Technology (IITDelhi) in 1999 and her Ph.D. degree in

2004 from the University of Maryland, College Park. She is a
professor of electrical
and computer engineering, and (by courtesy) of mathematics, at Iowa State
University. Her research interests lie at
the intersection of statistical machine
learning/data science, computer vision,
and signal processing. Her recent
research has focused on practically useful
and provably correct solutions for
dynamic high-dimensional data problems
such as dynamic compressive sensing
(CS) and robust principal component
analysis. She received the 2014 IEEE
Signal Processing Society Best Paper
Award for her modified-CS work that
appeared in IEEE Transactions on Signal
Processing in 2010. She is the special
issues area editor of IEEE Signal Processing Magazine, has served twice as an
associate editor of IEEE Transactions on
Signal Processing, and is the lead guest
editor of a Proceedings of the IEEE Special Issue on Rethinking PCA for Modern Data Sets.
sP

2019-2020

IEEE-USA Government
Fellowships
Congressional Fellowships
Seeking U.S. IEEE members interested in
spending a year working for a Member of
Congress or congressional committee.

Engineering & Diplomacy Fellowship
Seeking U.S. IEEE members interested in
spending a year serving as a technical adviser
at the U.S. State Department.

USAID Fellowship
Seeking U.S. IEEE members who are interested
in serving as advisors to the U.S. government
as a USAID Engineering & International
Development Fellow.

The application deadline for 2019-2020
Fellowships is 14 December 2018.
For eligibility requirements and application information, go to
https://ieeeusa.org/advocacy/government-fellowships/
or contact Erica Wissolik by emailing
e.wissolik@ieee.org or by calling +1 202 530 8347.

https://www.ieeeusa.org/advocacy/government-fellowships/

Table of Contents for the Digital Edition of IEEE Signal Processing - July 2018