Signal Processing - May 2017 - 89

learning are two extremes. Weakly
supervised learning occurs most frequently in our daily applications. The
design and analysis of a weakly supervised learning mechanism based on
CNNs is interesting and practical.
The interpretation of a CNN as a
guided multilayer RECOS transform
should be valuable to the investigation of
these topics.

Summary
The operating principle of CNNs was
explained as a guided multilayer RECOS
transform in this note. A couple of illustrative examples were provided to support this claim. Several known facts were
interpreted accordingly, and some open
issues were pointed out at the end.

Acknowledgments
I would like to thank Andrew Szot,
Shangwen Li, Zhehang Ding, and Gloria
Budiman for their help in running experi-

ments and drawing figures for this work. I
am also grateful for the valuable feedback
from friends, including Bart Kosko, Kyoung Mu Lee, Sun-Yuan Kung, and JenqNeng Hwang. This material is based
on research sponsored by the Defense
Advanced Research Projects Agency
(DARPA) and Air Force Research Laboratory (AFRL) under agreement number
FA8750-16-2-0173.
The U.S. Government is authorized
to reproduce and distribute reprints for
Governmental purposes notwithstanding any copyright notation thereon. The
views and conclusions contained herein
are those of the author and should not
be interpreted as necessarily representing the official policies or endorsements,
either expressed or implied, of DARPA
and AFRL or the U.S. Government.

Author
C.-C. Jay Kuo (cckuo@ee.usc.edu) is a
professor of electrical engineering at

the University of Southern California,
Los Angeles.

References

[1] C.-C. J. Kuo, "Understanding convolutional neural
networks with a mathematical model," J. Vis.
Commun. Image Represent., vol. 41, pp. 406-413,
Nov. 2016.
[2] W. S. McCulloch and W. Pitts, "A logical calculus
of the ideas immanent in nervous activity," Bull.
Math. Biophys., vol. 5, no. 4, pp. 115-133, 1943.
[3] F. Rosenblatt, "The perceptron: A probabilistic
model for information storage and organization in the
brain," Psychol. Rev., vol. 65, no. 6, pp. 386-408, 1957.
[4] G. Cybenko, "Approximation by superpositions of
a sigmoidal function," Math. Control Signals Syst.,
vol. 2, no. 4, pp. 303-314, 1989.
[5] K. Hornik, M. Stinchcombe, and H. White,
"Multilayer feedforward networks are universal
approximators," Neural Netw., vol. 2, no. 5, pp. 359-
366, 1989.
[6] K. Fukushima, "Neocognitron: A self-organizing
neural network model for a mechanism of pattern recognition unaffected by shift in position," Biol. Cybern.,
vol. 36, pp. 193-202, 1980.
[7] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner,
"Gradient-based learning applied to document recognition," Proc. IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
[8] A. K. Jain, "Data clustering: 50 years beyond
k-means," Pattern Recogn. Lett., vol. 31, no. 8, pp.
651-666, 2010.

Rodrigo Capobianco Guido

Effectively Interpreting Discrete Wavelet Transformed Signals

ollowing two decades of research
focusing on the discrete wavelet transform (DWT) and driven by students'
high level of questioning, I decided to
write this essay on one of the most significant tools for time-frequency signal
analysis. As it is widely applicable in a
variety of fields, I invite readers to follow
this lecture note, which is specially dedicated to show a practical strategy for the
interpretation of DWT-based transformed
signals while extracting useful information from them. The particular focus
resides on the procedure used to find the
time support of frequencies and how it is
Digital Object Identifier 10.1109/MSP.2017.2672759
Date of publication: 26 April 2017

influenced by the wavelet family and the
support size of corresponding filters.

Relevance
Frequently, DWT computation is much
faster than that of the discrete Fourier
transform (DFT) and even the fast Fourier transform (FFT), encouraging its
usage. Furthermore and opposite to the
DFT, to the FFT, and even to the shorttime Fourier transform, the DWT reveals
the time support of frequencies efficiently, as described in [1]. Thus, its study is of
paramount importance.

Prerequisites
On one hand, essential knowledge of
digital filters and wavelet families is only
IEEE Signal Processing Magazine

May 2017

desirable. On the other hand, the basic
aspects involving the procedures used to
calculate DWTs from discrete-time signals are imperative for the full comprehension of this lecture note. If readers are not
comfortable with the topic, I encourage
you to consult [1]-[3], written in a friendly
manner, before proceeding any further.

Problem statement and solution
Problem statement
Given the pair of signals s [·] and y [·],
both of size M equal to a power of two
and indexed from 0 to M - 1, being
the former and the latter the discretetime input and its DWT, respectively,
the problem is to extract and interpret
89

Table of Contents for the Digital Edition of Signal Processing - May 2017