IEEE Circuits and Systems Magazine - Q3 2020 - 9

and port their code in order to be executed by scarce
resource devices. The second approach bets on integrating ML libraries within MCUs, hence providing them with
local training and analysis capabilities. This permits to
generate models from data retrieved by the own device,
which may improve the accuracy of the model and also
enables the implementation of unsupervised learning algorithms. Finally, the last approach relies on the availability of a dedicated co-processor that supports the main
computing unit on ML-specific tasks. This strategy permits to enhance the computation performance, although
it is the less common approach as the price and complexity of the processing platforms are notably increased.
Big technological companies are promoting the expansion of the TinyML ecosystem by offering open-source
development libraries. Google has launched TensorFlow
Lite [35], which includes a set of tools to adapt TensorFlow models aiming at making them to be runnable in mobile and embedded devices. This framework is composed
of two main elements, namely, the converter, which ports
TensorFlow models to optimized code that can be executed in constrained platforms, and the interpreter, which
actually runs the code generated by the converter. The
optimized models, which cover a number of algorithms
from the NN family, can be executed in a range of platforms such as smart-phones, embedded Linux and MCUs.
Focusing on the latter, the optimized code is provided in
C++ 11 and requires 32-bit processors to be run. It has
been successfully tested in devices with ARM Cortex-M
series processors, e.g., Arduino nano, and other architectures such as ESP32. Given the notoriety of Arduino, a
specific library for this platform is available through its
IDE. Besides the development of TensorFlow Lite, another
related Google's initiative has been the adaptation of the
GO programming language to microcontrollers [36]. In
turn, Microsoft has also contributed to the TinyML scene
by releasing its open source Embedded Learning Library
(ELL) [37]. This framework permits to design and deploy
pre-trained ML models onto constrained platforms, e.g.,
ARM Cortex-A and Cortex-M architectures such as Arduino, Raspberry Pi, and micro:bit. ELL can be understood
as an optimizing cross-compiler that runs on a regular
desktop computer and outputs C++ code ready to be executed on the targeted single-board computer. ELL's API
can be employed either from C++ or Python and makes
use of pre-trained NN models produced by the Microsoft
Cognitive Toolkit (CNTK) [38], Darknet [39], or in Open
Neural Network Exchange (ONNX) format [40].
ARM is also pushing for the integration of ML in their
devices. The ARM-NN toolkit [41] is part of Linaro's ML
Initiative [56] and allows to integrate NN workloads within
ARM Cortex-A CPUs, ARM Mali GPUs, and Ethos Neural
Processing Units (NPUs). ARM-NN is an open-source projTHIRD QUARTER 2020

ect compatible with existing ML frameworks such as TensorFlow and Caffe [57], and the ONNX format. As ARM-NN
does not provide support to the Cortex-M CPUs, ARM has
also released the Cortex Microcontroller Software Interface Standard-NN (CMSIS-NN) [42], which is a collection of
NN kernels optimized to be executed on this processor series. CMSIS-NN is able to convert floating point-based NNs
generated by popular libraries, e.g., TensorFlow, PyTorch,
and Caffe, into fixed-point format that can run on Cortex-M
devices. Another electronics manufacturer that has developed specific libraries for its devices is STMicroelectronics. Concretely, the STM32 Cube.AI toolkit [43] permits
to integrate pre-trained NNs within STM32 ARM CortexM-based microcontrollers. It generates STM32-compatible
C code from NN models generated by Tensorflow and
Keras [58], or in the standard ONNX format. As an interesting feature, STM32Cube.AI permits to run large NNs by
storing weights and activation buffers in external flash
memory and RAM, respectively.
Besides open-source initiatives, some institutions
and companies have also launched licensed products.
The Fraunhofer Institute for Microelectronic Circuits and
Systems (IMS) has developed the Artificial Intelligence
for Embedded Systems (AIfES) library [44], which is a
C-based and platform-independent tool for generating
NNs compatible with a range of MCUs, e.g., Arduino UNO,
ATMega32U4, or STM32 F4 Series. It can also produce efficient code for Windows in the form of Dynamic Link Library (DLL) and Linux platforms such as Raspberry Pi.
In its current version, it is able to extracts all parameters
from models produced by TensorFlow or Keras. Different from the frameworks reviewed above, AIfES permits
to implement the NN training process on the embedded
device, i.e., it belongs to the second family of frameworks
identified above. Cartesiam's NanoEdge AI Studio [45] is
another commercial product which permits to create ML
static libraries to embed them in Cortex-M-based MCUs.
Similar to AIfES, it allows to integrate the training process
within the constrained device but, in addition, it also
permits to run unsupervised learning algorithms on the
MCU. As a part of this toolkit, a microcontroller emulator is provided in order to test and debug the generated
model prior to its deployment on the device.
Surprisingly, all the frameworks presented so far only
focus on a single type of ML algorithm, namely, NN. However, different initiatives released by makers, developers,
and researchers also consider other ML techniques such
as Naive Bayes classifier, decision trees, k-Nearest Neighbors (k-NN), among others. MicroML [46] is a project that
permits to port Support Vector Machine (SVM) and Relevance Vector Machine (RVM) algorithms to C code that
can run in a number of MCUs, e.g., Arduino, ESP8266,
ESP32, and others with C support. It interoperates with the
IEEE CIRCUITS AND SYSTEMS MAGAZINE

IEEE Circuits and Systems Magazine - Q3 2020

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q3 2020