IEEE Consumer Electronics Magazine - September/October 2021 - 104

AI-Driven Security Solutions for the Internet of Everything
DataFrames of training and testing DataFrames.
Then, the TF-IDF DataFrames are used
for the training and prediction of the machine
learning model. The Elastic Net regularization
is used to train the logistic regressionmodel.
Figure 2. Two-tier intrusion detection framework for
embedded systems.
The proposed framework uses a set of standardized
methods based on Spark DataFrame to
combine multiple algorithms into one machine
learning pipeline. A Pipeline can link multiple
Transformers and Estimators together. The relevant
terms are described as follows.
DataFrame: The Spark DataFrame can contain
many columns and store different data
types.
Transformer: The Spark Transformer can
transform one DataFrame into another. The
Transformer implements the transform()
method.
Estimator: The Spark Estimator can be applied
to DataFrames to generate Transformers. The
Estimator implements the fit() method.
Pipeline: In machine learning, data are usually
processed by a set of algorithms. This
kind of machine learning workflow is named
pipeline. By using pipeline, the entire
machine learning process can be created and
adjusted. A Pipeline can link multiple Transformers
and Estimators together.
PROCESSES OF THE PROPOSED
FRAMEWORK
The framework proposed in this article
includes the following steps.
1) Preparation of DataFrames. The DataFrames for
training and testing/prediction are generated.
2) Constructing machine learning pipeline for
training and prediction. A machine learning
pipeline is constructed to generate TF-IDF
(term frequency-inverse document frequency)
104
3) The model selection method is used to adjust
the machine learning pipeline. Model selection
is an advanced computing paradigm of
Spark. Model selection can adjust the entire
pipeline at once. In this step, the machine
learning pipeline model constructed in the
previous step is improved. The model selection
is carried out by using " cross-validation "
and " training-validation split. " The detection
accuracies of the two model selection methods
are tested.
Preparation of DataFrames for Training and
Testing
This article uses ADFA-LD to represent system
call traces generated from embedded devices.
The ADFA Linux dataset (ADFA-LD)10 was
created using the Ubuntu Linux operating system,
which was attacked by a variety of recent
penetration testing approaches. ADFA-LD
includes 833 normal traces and 746 attack
traces with six attack approaches, namely,
" brute force password guessing, " " add new
superuser, " " Java Meterpreter payload, " " Linux
Meterpreter payload, " and " C100 webshell. "
Algorithm 1. Constructing Machine Learning
Pipeline for Training and Prediction
val pipeline ¼ new PipelineðÞ
:setStagesðArrayðtokenizer;
ngram; hashingTF; idf; lrÞÞ
val model ¼ pipeline:fitðtrainingDataÞ
val predictions ¼ model:transformðtestingDataÞ
In the proposed framework, the ADFA-LD
dataset is stored in the Google cloud storage system.
After the attack traces and normal traces of
ADFA-LD are loaded from the cloud storage system,
the duplicate traces are deleted. The training
traces and testing traces are randomly
selected (60% and 40%), and the DataFrames for
training and testing are generated (trainingData
and testingData in the Scala code).
IEEE Consumer Electronics Magazine
IEEE Consumer Electronics Magazine - September/October 2021

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - September/October 2021
