IEEE Consumer Electronics Magazine - July/August 2021 - 14

Consumer Electronics Security
Table 2. Summarized performance results.
Method
Type
Original
DRAMNet
Augmented
Original
AlexNet
Augmented
Original
VGGNet
Augmented
SGD
91.23
95.50
91.89
96.93
91.49
94.72
Accuracy %
Recall %
Adam
94.35
98.84
96.49
98.69
95.44
97.20
SGD
89.56
96.69
91.52
96.81
91.07
94.68
Adam
94.21
98.81
95.39
98.23
94.89
96.95
Precision %
SGD
89.29
96.06
89.37
96.66
88.73
94.34
Adam
94.17
98.73
96.41
98.51
94.73
96.88
F-score %
SGD
88.42
96.91
90.43
96.73
89.88
94.50
Adam
94.19
98.64
95.89
98.36
94.97
96.91
Google. For our experiments, 180 different measurements
were collected from three different
DRAMs based on their power-up values. We specified
60% of the total measurement data (DRAM
images) for training DRAMNet and the rest 40%
for the testing/evaluation phase. Note that for
training data, we ensured that the percentage of
DRAM images selected from each particular
DRAM was equal. In order to minimize the error
rate of the cost function, two different optimization
methods, i.e., adaptive moment estimation
(Adam) and stochastic gradient descent (SGD)
are applied. Since our network is used as a classifier
for DRAM authentication with three classes
(three DRAMs), SoftMax function is used to predict
the probability of each DRAM input.13
Evaluation Metrics for Comparison
To evaluate the effective performance of our
proposed method, the following four metrics are
utilized: accuracy, precision, recall, and F-score.
These four metrics are the most frequently used
performance indicators for machine learning
models. The following equation shows how
these metrics are derived:
Accuracy ¼ TPþTNþFPþFN
TP
TPþTN
Precision
Recall
¼ TPþFP
TP
¼ TPþFN
F1 Score ¼ 2 PrecisionRecall
PrecisionþRecall
(1)
where TP, TN, FP, and FN represent the number of
true positives, true negatives, false positives, and
false negatives, respectively. Moreover, we have
also calculated the receiver operation characteristic
14
(ROC) curves. These curves show the tradeoff
between false positive rate (FPR) and true positive
rate (TPR). FPRrefers to the rate atwhicha classifier
incorrectly matches the nonmatching data (outlier)
to the target class. TPR refers to the rate that a classifier
correctly matches the matching data (target)
to the target class.
Classification Results and Discussion
Table 2 presents the summarized performance
results of the three CNN models, where
DRAMNet refers to our CNN model, and AlexNet
and VGGNet refer to other developed CNN architectures.
First, the training data were conducted
without augmentation. In this case, the DRAMNet
CNN model achieved 91.23% average accuracy,
89.56% recall, 89.29% average precision,
and 88.42% F-score based on the SGD optimizer.
Since our DRAMNet pool is not large enough to
achieve the optimal performance, training
DRAMNet data can be enlarged by augmenting
the DRAM images, which results in higher
authentication accuracy. Data augmentation is
one of the advantages of applying images as
input data into the deep learning technique. We
augmented DRAM images by cropping each
image in six different ways at the training model.
By using this augmentation, our DRAMNet model
achieved more than 6% higher accuracy, recall,
precision, and F-score. Surprisingly, the authentication
performance is increased by applying
the Adam optimizer technique for all the models.
Based on the results presented in Table 2, the
AlexNet CNN model exhibits the best accuracy,
recall, and sensitivity precision when we train
IEEE Consumer Electronics Magazine
IEEE Consumer Electronics Magazine - July/August 2021

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - July/August 2021
