Systems, Man & Cybernetics - July 2015 - 41

The existing texts on communication principles deal
with the theory and practical significance of special
types of networks commonly used in radio or telephone
circuits. Progressive research, however, demands a
more thorough grasp of the fundamental methods of
attack on network problems in general. [11, p. iii]
It can be assumed that not only had Guillemin actually
read Cauer's work in the original German but also that
there was a close teacher-pupil relationship between a
famous professor and his former student, a relationship
that should be researched somewhat more closely.
When Filtering and Predicting Fused
with Communication
In 1942, the text of Wiener's book The Extrapolation,
Interpolation, and Smoothing of Stationary Time
Series with Engineering Applications [Figure 2(a)] was
circulated in 300 copies of a classified memorandum.
In this text, Wiener pursued the goal of unifying communication engineering with the field of statistical time
series in theory and practice, and the relevant methods for this purpose included filtering, in addition to
extrapolation and prediction. Moreover, he fused these
methods "into a common technique which, in the opinion of the author, is more effective than other existing
techniques alone" [35, p. 9].
Predicting a time series (or a message) could certainly not simply consist of its constant continuation.
This would instead be a matter of statistical prediction,
estimating the continuation of the time series (or communication), its most probable future pattern while minimizing random error, as Wiener described the problem in
greater detail:
We have a message which is a time series, and a noise
which is also a time series. If we seek that which we
know concerning the message, which is not bound to a
specific origin in time, we shall see that such information
will generally be of a statistical nature; and this will likewise be true of our information of the same sort concerning the noise alone, or the noise and the message jointly.
While this statistical information will, in fact, never be
complete, as our information does not run indefinitely
far back into the past, it is a legitimate simplification of
the facts to assume that the available information runs
back much further into the past than we are called upon
to predict the future. The usual electrical wave filter
attempts to reproduce a message "in its purity," when the
input is the sum of a message and a noise. [35, p. 10]
Wiener (Figure 3) had been working with the filter problem on a very mathematically abstract level, yet he knew
very well that there was quite a lot to do below this level to
implement his standardization program. He thus referred to
the work of appropriate experts, naming his MIT colleague
Guillemin (Figure 3): "The problem of realization takes
one into the theory of equivalent networks as developed by
Guillemin and others" [35, p. 22].

The research conducted by Wiener and Guillemin
in the late 1940s would be consolidated by members of
a new generation of communication engineers, which
included, in particular, Zadeh. During his studies at
MIT, Zadeh encountered Wiener's Cybernetics [36] and
the paper "Mathematical Theory of Communication"
by Claude E. Shannon (1916-2001), an American mathematician and electrical engineer [Figure 4(a)] [31]. "An
Extension of Wiener's Theory of Prediction" was the
title of an internal report [39] written in 1949 by Zadeh,
who was then an assistant professor at Columbia University. Soon after finishing his Ph.D. dissertation, he
published the report in Journal of Applied Physics [40],
with his supervisor, Ragazzini [Figure 4(c)], as a coauthor [Figure 2(b) and (c)].
In this work, which expanded Wiener's prediction
theory, Zadeh indicated that the foundations for his
work could be found both with Wiener and with Andrei
N. Kolmogorov. Both Wiener and Kolmogorov had proceeded from this problem: if knowledge about the past
and present of the physical system is given, how can its
future be predicted? Zadeh generalized Wiener's theory
in two ways:
◆ The signal component of a given time series was separated into two parts, of which the first is a nonrandom
function in time that can be represented as a polynomial, while the other part functions as a stationary
and statistical random function represented by a given
correlation function. In Wiener's theory, by contrast, a
nonrandom portion of the signal occurred only when it
consisted of a known function in time.
◆ The response of the predicting system or the weighting
function used to make the prediction should disappear
outside of a finite-time interval. In Wiener's theory, on
the other hand, this time interval was assumed to be
infinitely long.
Zadeh showed that determining the weighting function
leads to the solution of a modified Wiener-Hopf equation,

Figure 3. Wiener (left) and Guillemin (right) were
colleagues at MIT. (Photo courtesy of the MIT Museum.)

Ju ly 2015

IEEE Systems, Man, & Cybernetics Magazine

Table of Contents for the Digital Edition of Systems, Man & Cybernetics - July 2015