IEEE Circuits and Systems Magazine - Q3 2020 - 15

training samples does not impact on the weight of the
produced model, the 80% of the dataset has been employed for training and the other 20% for test. The obtained outcomes are shown in Table V. It can be seen
that in the MLP case the limiting factor is the Arduino
Uno's SRAM, as the network's weights and biases are
stored there. Thus, when the number of neurons per layer is 10, the maximum number of hidden layers for not
consuming all the unit's SRAM is 3. Regarding the algorithm's performance, a trade-off between accuracy and
network complexity is evidenced. However, employing
one hidden layer with 4 neurons permits an accuracy of
0.97, which is the best figure for this simple configuration. It is clear the direct relationship between the overall number of neurons in the network and the processing time for each classification operation, which should
be also considered when selecting the most adequate
system configuration. However, observe that the times
employed by the MLP classifier are notably shorter than
those needed by the MicroMLGen's SVMs. The achieved
accuracy is also better for the MLP algorithm.
The adaptation of other well-known ML mechanisms
such as decision trees and RF has been evaluated as well.
To this end, we have employed the sklearn-porter and emlearn frameworks to port the models generated by Scikitlearn. In this case, sklearnporter produced suitable code
for Arduino Uno, given that decision trees and RF are lighter than SVMs. A similar performance analysis to those
presented previously is shown in Table VI. The dataset
has been split as in the case of MLP evaluation, i.e., 80%

for training and 20% for evaluating the produced classifier.
Observe how these algorithms capture the logic employed
for labelling our synthetic dataset, which is evidenced by
the good accuracy obtained. By increasing the depth of the
decision tree, we improve the classifier performance up to
a perfect accuracy of 1 with decision trees of 6 levels. Regarding the RF classifier, we have evaluated one model consisting of 10 estimators (trees) and we also obtain a perfect
accuracy although the memory footprint and processing
time are notably greater for this scheme. Comparing both
frameworks, sklearnporter provides faster and lighter decision trees in comparison with emlearn. However, the opposite behaviour is obtained for the RF algorithm.
In the light of the obtained results, the performance
of decision trees and RF are notably better than that
of SVM and MLP for solving the intelligent communication interface selection issue under consideration. They
achieve greater accuracy with reduced memory footprint and computation delay. Therefore, for the specific
case of smart multi-RAT configuration management,
this family of algorithms would be the most adequate to
be integrated within the MCU.
To sum up, it can be concluded that great advances have
been made so far for integrating ML algorithms within frugal devices. These achievements were unimaginable few
time ago and they pave the way for the arrival of smart applications and services that will not require a continuous
processing support from the cloud. However, we are just in
the inception of this movement and more efforts are needed for the integral development of the TinyML ecosystem.

Table IV.
SVM generated by MicroMLGen.
Samples/Kernel

Linear

Polynomial

rbf

250

Accuracy = 0.7816
F1 = 0.7119
Memory = 10678 B
Time = 11.1±0.2 ms
Accuracy = 0.7871
F1 = 0.7161
Memory = 17782 B
Time = 22.3±0.1 ms
Accuracy = 0.7922
F1 = 0.723
Memory = 25298 B
Time = 34.1±0.2 ms
Accuracy = 0.7921
F1 = 0.7232
Memory = 30156 B
Time = 41.8±0.2 ms
Accuracy = 0.7939
F1 = 0.725
Memory = 37930 B
(out-of-range)

Accuracy = 0.7992
F1 = 0.7643
Memory = 10264 B
Time = 73.7±0.3 ms
Accuracy = 0.8552
F1 = 0.8074
Memory = 15636 B
Time = 73.2±0.7 ms
Accuracy = 0.8561
F1 = 0.8347
Memory = 20642 B
Time = 107.2±0.3 ms
Accuracy = 0.8688
F1 = 0.8443
Memory = 25530 B
Time = 140.1±0.2 ms
Accuracy = 0.8757
F1 = 0.845
Memory = 31968 B
Time = 183.6±0.4 ms

Accuracy = 0.7355
F1 = 0.6298
Memory = 12172 B
Time = 34.7±0.03 ms
Accuracy = 0.7472
F1 = 0.6661
Memory = 20574 B
Time = 72±0.047 ms
Accuracy = 0.75
F1 = 0.6815
Memory = 26644 B
Time = 98.9±0.07 ms
Accuracy = 0.7757
F1 = 0.7402
Memory = 35184 B
(out-of-range)
Accuracy = 0.7818
F1 = 0.7484
Memory = 44058 B
(out-of-range)

500

750

1000

1250

THIRD QUARTER 2020

IEEE CIRCUITS AND SYSTEMS MAGAZINE

IEEE Circuits and Systems Magazine - Q3 2020

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q3 2020