IEEE Consumer Electronics Magazine - January/February 2020 - 70

Column: The Art of Storage

AI Inference and Storage
Tom Coughlin
Coughlin Associates, Inc.

& MOST

for artificial intelligence (AI) applications is done in large data centers using curated training data. This is because
the computing and energy resources required
are large. As a recent paper points out, "it was
found that AI models that use neural architecture search emit the carbon dioxide equivalent
of nearly five times the lifetime emissions of an
average American car."1
However, the application of trained AI at
the edge of networks or at endpoints involves
inference engines. These inference engines are
embedded semiconductor devices, which store
the weighting functions from the AI training for
AI applications in the field that often work in
harsh constrained environments with limited
energy supplies. As a consequence, large-scale
use of AI applications requires new levels of integration and technologies to reduce energy
consumption.
One way to reduce device energy consumption
is to replace volatile memory with nonvolatile
memory. In particular, replacing static random
access memory (SRAM) memory with fast nonvolatile memory could save energy and, since SRAM
takes up a lot of real-estate on the die, could also
increase the amount of memory that can be put
into a given area on a semiconductor die.2
Not Or (NOR) flash memory is often used as a
nonvolatile memory in embedded applications,
MACHINE LEARNING

Digital Object Identifier 10.1109/MCE.2019.2940743
Date of current version 6 December 2019.

2162-2248 ß 2019 IEEE

but NOR flash faces limitations on how small it
can get. NOR flash probably cannot be economically manufactured with lithographic features
less than about 28 nm. As the memory requirements for AI-enabled applications increases,
NOR flash built into embedded devices faces
severe limitations.
A good candidate to replace slower SRAM
and NOR flash for embedded applications is magnetic random access memory (MRAM).3 MRAM
is nonvolatile and maintains its data even when
the power is turned OFF. Current spin tunnel torque MRAM has very high write endurance (up to
1015) and read and write times that can match
slower SRAM used in higher level caching applications (e.g., L3 and L4).
Being able to turn a device OFF when it is not
in active use, and turn it ON and being able to
pick up where the machine left OFF, can save considerable energy and enable battery powered
applications. This could help with many mobile
and remote AI-powered consumer and industrial
products.
Furthermore, while SRAM memory cells
have six to eight transistors per cell, MRAM
has one-transistor per memory cell. Thus, for a
given die area, MRAM can provide 10Â or
greater the memory than SRAM. An embedded
chip with MRAM replacing some of the slower
MRAM and embedded NOR flash is depicted in
Figure 1. The power savings and memory density increase possibly by using MRAM in
embedded products.

Published by the IEEE Consumer Electronics Society

IEEE Consumer Electronics Magazine

IEEE Consumer Electronics Magazine - January/February 2020

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - January/February 2020