IEEE Spectrum - North American - March 2017 - 36

Input layer

Hidden layer 2

Hidden layer 1
36

MAR 2017

NORTh AMERICAN

types of signals it would later have to
classify on the fly.
Then we used samples of noisy speech
and their corresponding results on the
ideal binary mask to complete the second phase of training, which was the
supervised learning. In particular, the
set of 1s and 0s that make up the ideal
binary mask was like an answer sheet
that we used to test and improve our program's ability to separate speech from
noise. For each new sample, the program
would extract a set of attributes from
the noisy speech. Then, after analyzing
these attributes-frequencies, intensities,
and so on-the filter performed a provisional classification-was it speech? was it
noise?-and compared the result to what
the ideal binary mask would determine
in the same situation. If the result was
different from the 1s and 0s within our
perfect binary mask filter, we tweaked
the neural network's parameters accordingly, so that the network would produce
results closer to the 1s and 0s of the ideal
binary mask on its next try.
To make these adjustments, we first calculated the error of the neural network,
measured as the discrepancy between the
ideal binary mask and the result at the neural network's final layer, which is known as
the output layer. Once we computed this
error, we would then use it to change the
weights of the neural network's connections so that if the same classification was
carried out again, the discrepancy would
be reduced. The training of the neural network consisted of performing this procedure thousands of times.
One important refinement along the
way was to build a second deep neural
network that would be fed by the first
one and fine-tune its results. While
Output
layer

Hidden layer 3
|

SPECTRUM.IEEE.ORG

sMart layers: A deep
neural network consists
of two or more processing layers in between
the input layer, through
which information is fed
into the system [left], and
the output layer, which
reveals the results [right].
To improve performance,
researchers can adjust
the system's parameters
and tweak the connections between layers.

1880 to 1920

1921 to 1953

1984 to present

that first network had focused on labeling attributes within each individual
time-frequency unit, the second network would examine the attributes of
several units near a particular one. In
other words, the second network provided the first network with extra context
about the speech and noise it processed
and further improved its classification
accuracy. For example, a syllable may
span many time-frequency units, but the
background noise could change abruptly
while it was being spoken. In our case,
having contextual clues could help the
program to more accurately separate
speech from noise within the syllable.
At the end of the supervised training, the deep-neural-network classifier proved to be far superior to earlier
methods at separating speech from
noise. In fact, this algorithm was the
first, of any technique relying on monaural techniques, to achieve major
improvements in hearing-impaired listeners' ability to make sense of spoken
phrases obscured by noise.
To test it with human subjects, we
asked 12 hearing-impaired people and
12 with normal hearing to listen through
illustration by

Erik Vrielink

From Top: hearing aiD museum; Joe haupT/Flickr; sspl/geTTy images

Neural networks come in many shapes
and sizes and with varying degrees of
complexity. Deep neural networks are
defined as having at least two "hidden"
processing layers, which are not directly
connected to a system's input or output.
Each hidden layer refines the results fed
to it by previous layers, adding in new
considerations based on prior knowledge.
For example, a program designed
to verify a customer's signature might
begin by comparing a new signature to a
sample included in a training database.
However, that program also knows from
its training that the new signature does
not need to precisely match the original.
Other layers can determine if the new
signature shares certain qualities that
tend to remain consistent in a person's
signature, such as the angle of slant, or
the failure to dot the letter i.
To build our own deep neural network, we began by writing algorithms
to extract features that could distinguish
voices from noise based on common
changes in amplitude, frequency, and
the modulations of each. We identified
dozens of attributes that could help our
program discriminate between speech
and noise to some extent, and we used
all 85 of them to make the algorithms as
powerful as possible. Among the most
important attributes we identified were
the frequencies of the sounds and their
intensities (loud or soft).
Next, we trained the deep neural network to use these 85 attributes to distinguish speech from noise. This training
occurred in two phases: First, we set the
program's parameters through unsupervised learning. This means we loaded
many examples of the attributes into
the program in order to prime it for the

http://SPECTRUM.IEEE.ORG

Table of Contents for the Digital Edition of IEEE Spectrum - North American - March 2017