IEEE Spectrum February, 2016 - 42

Hidden
layer

Input
bottoM's uP: a standard feed-forward network has the input at the bottom.

the base layer feeds into a hidden layer, which in turn feeds into the output.

Output

Hidden
layer

Input
looP the looP: a recurrent neural network includes connections between neurons

in the hidden layer [yellow arrows], some of which feed back on themselves.

Output

Hidden
layer

Input
Time step 1

Time step 2

tiMe after tiMe: the added connections in the hidden layer link one time step
with the next, which is seen more clearly when the network is "unfolded" in time.

The neural network learns proper nouns like "Coors Light" and beer
jargon like "lacing" and "snifter." It learns to spell and to misspell, and
to ramble just the right amount. Most important, the neural network
generates reviews that are contextually relevant. For example, you can
say, "Give me a 5-star review of a Russian imperial stout," and the software will oblige. It knows to describe India pale ales as "hoppy," stouts
as "chocolatey," and American lagers as "watery." The neural network
also learns more colorful words for lagers that we can't put in print.
This particular neural network can also run in reverse, taking any
review and recognizing the sentiment (star rating) and subject (type
of beer). This work, done by one of us (Lipton) in collaboration with
his colleagues Sharad Vikram and Julian McAuley at the University of
California, San Diego, is part of a growing body of research demonstrating the language-processing capabilities of recurrent networks. Other
42

FEB 2016

north amErican

SPEctrUm.iEEE.orG

related feats include captioning images, translating foreign languages, and even answering
e-mail messages. It might make you wonder
whether computers are finally able to think.
That's a goal computer scientists have pursued for a long time. Indeed, since the earliest
days of this field, they have dreamed of developing truly intelligent machines. In his 1950
paper, "Computing Machinery and Intelligence," Alan Turing imagined conversing with
such a computer via a teleprinter. Envisioning what has since become known as a Turing
test, he proposed that if the computer could
imitate a person so convincingly as to fool a
human judge, you could reasonably deem it
to be intelligent.
The very year that Turing's paper went to
print, Gnome Press published I, Robot, a collection of Isaac Asimov's short stories about
intelligent humanoids. Asimov's tales, written before the phrase "artificial intelligence"
existed, feature cunning robots engaging in
conversations, piloting vehicles, and even
helping to govern society.
And yet, for most of the last 65 years, AI's
successes have resembled neither Turing's
conversationalists nor Asimov's humanoids.
After alternating periods of overenthusiasm
and subsequent retrenchment, modern AI
research has largely split into two camps. On
one side, theorists work on the fundamental mathematical and statistical problems
related to algorithms that learn. On the other
side, more practically oriented researchers
apply machine learning to various real-world
tasks, guided more by experimentation than
by mathematical theory.
Until recently, both sides of this divide
focused on simple prediction problems. For
example: Is an e-mail message spam or not
spam? Or: What's the probability that a loan
will default? A cynic might say that we dreamed
of creating humanlike intelligence and got
spam filters instead. However, breakthroughs
in neural-network research have revolutionized computer vision and natural-language
processing, rekindling the imaginations of the
public, researchers, and industry.
The modern incarnation of neural networks,
commonly termed "deep learning," has also widened the gap between theory and practice. That's
because, until recently, machine learning was
dominated by methods with well-understood
theoretical properties, whereas neural-network
research relies more on experimentation.

Zachary c. Lipton/University of caLifornia, san Diego (2)

Output

http://googleresearch.blogspot.com/2015/11/computer-respond-to-this-email.html http://googleresearch.blogspot.com/2015/11/computer-respond-to-this-email.html http://www.loebner.net/Prizef/TuringArticle.html http://www.loebner.net/Prizef/TuringArticle.html https://en.wikipedia.org/wiki/Deep_learning http://SPEctrUm.iEEE.orG

Table of Contents for the Digital Edition of IEEE Spectrum February, 2016