IEEE Consumer Electronics Magazine - April 2017 - 55

Although speed and power consumption are critical to end
products, people often design their networks using slightly
slower but higher-level frameworks. This allows us to test out
ideas and prove products before creating highly optimized
implementations in hardware or software. Although deep
learning can be implemented in nearly any programing language, most resources and community support available is
intended for Python. Although most deep learning tools are
now compatible with Python 3X, the deep learning community still primarily uses Python 2.7.
The most popular high-level frameworks for this are
Caffe, Theano, and TensorFlow. Caffe [18] was created at the
Berkely Vision and Learning Center. Models are defined in a
configuration file instead of being coded directly. They claim
to have the fastest CNN implementation available. Theano
[19] was initially designed for fast, stable symbolic operations, including symbolic differentiation. It dynamically generates optimized C code, which can be transparently executed
on the GPU up to 140-times faster than an equivalent CPU
implementation. Finally, TensorFlow [20], the newest of the
three, was designed by Google and is currently experiencing
the fastest growth in usage. As of this writing, TensorFlow
has slightly slower performance than the other two on benchmarks, but it is easier to deploy on multiple GPUs.
There are a number of frameworks built on top of these
platforms, such as Lasagne and Keras, that allow further
abstraction and thus ease and speed of development. Keras is
a good choice, especially for beginners, because it can use
either TensorFlow and or Theano as a back end and it provides a simpler model for development.

exaMple applIcatIons
Recently, Google provided a mobile TensorFlow application
program interface (API) that allows developers to deploy
deep learning models directly to smartphones [21]. Google
uses it for Google Translate's instant visual translation and
recommends it for any application where processing needs to
be done on the device.
FotoNation (www.fotonation.com), a company with software running in over 2.7 billion smartphones worldwide,
has released a new face-recognition product based on the
latest deep learning techniques to achieve rapid and accurate face recognition. Nuance uses deep learning for its
Dragon Naturally Speaking line of voice recognition applications [22]. Deep learning is used in both initial product
development and later for custom refinement on consumer
electronics. Prisma, with 27 million users, is a popular cell
phone app that uses deep learning to transform photos into
paintings in the style of famous artists [23]. Soundhound,
Inc., is collaborating with Nvidia using deep learning "to
add a smart, voice-enabled, conversational interface to
every technology that humans interact with" based on its
platform called "Houndify" [24].
At Apple's Worldwide Developers Conference 2016,
Apple's John Gruber revealed in his keynote speech that it
uses deep learning in its photo app, first in its own data center

where it pretrains a model based on a vast base of labeled
photos, and later on the iPhone to annotate images as they are
taken. It also analyzes all photos on a phone when the device
is plugged in and not in use so as to avoid draining battery
power. Google takes a different approach, avoiding on-device
classification, instead processing users' photos with its cloud
infrastructure [25].
Neurala released an API that uses deep learning to perform real-time object tracking, recognition, and other tasks.
As of September 2016, it is the largest supplier of deep learning software for consumer drones. Its deep learning software
allows drones to operate autonomously, performing tasks
such as "follow this person" or "find this car." Such sophisticated task performance was within the realm of sci-fi only a
few years ago [26].

ConClusion
Deep learning is being used today in our cell phones, cars,
and tablets and computers. It has pushed the boundaries of
what is possible for tasks such as image segmentation [13],
object detection [27], face recognition [28], voice analyzing
[29], emotion detection [30], and gender recognition [31].
Why has deep learning suddenly catalyzed research across so
many fields? It is a combination of many factors: the recent
emergence of highly affordable high-density, GPU-based
computational hardware has provided the engines to process
very large data sets and implement the advanced training
methodologies required to develop accurate CNNs; the widespread availability of GPUs in today's devices, coupled with
cloud-based data processing services, provides the means to
apply these CNN architectures to everyday applications such
as voice or image processing; big data provides the fuel to
drive research activity and refine results to the point where
deep learning solutions typically outperform even the best of
human-designed pattern recognition tools.
Today, we are at a point where many new problems can be
tackled through the use of deep learning techniques. Many of
these are long-standing problems, such as voice recognition,

Figure 11. the Movidus fathom uSB stick. (image courtesy of Movidius.)

APRIL 2017

IEEE Consumer Electronics Magazine

http://www.fotonation.com

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - April 2017