IEEE Consumer Electronics Magazine - January/February 2022 - 55

Table 1. Cluster FM-Scores With J1939 Data.
Clustering algorithm
Agglomerative hierarchical (AH)
Genetic algorithm (GA)
DBSCAN
EM-GMM
FM-score
71.72%
41.03%
54.90%
71.82%
labels, and contains over 300 CAN IDs. We created
the distance matrix for this data and passed
it to the four clustering algorithms.
To measure the performance of the clustering
algorithms, we manually generated sets of
related PGNs to define the IDs that belong to similar
vehicular functions. We filtered out some
PGNs that are not well-defined and omit them
from calculation of performance. Performance is
calculated by comparing the manually labeled
ground truth and each algorithm's generated
clusters using the Fowlkes-Mallows score (FMScore).12
FM-Score is an extrinsic clustering evaluation
metric that evaluates the similarities
between two clusters, i.e., the ground truth and
each algorithm's output. Table 1 summarizes the
results of the FM-Scores for each of the four clustering
approaches we investigated.
The AH and EM-GMM algorithms performed
the best, and we focus further on them to understand
the benefits of clustering. With this data,
which is much richer, the dendrogram is less
useful, so we visualized the high-dimensional
clusters using principal component analysis
(PCA) to help guide our parameter tuning. An
example of the case using AH clustering is shown
in Figure 6. The AH algorithm returned four clusters
(0-3) as shown in Table 2. The largest cluster
contained all the driving critical functions, such
as braking and engine controller. The other
three clusters were much smaller and contained
Table 2. AH Clusters Found in J1939 Data.
Cluster label
1
2
3
CAN function
Driving (engine and brake)
Proprietary
Memory access
Data transfer
proprietary IDs, memory accesses, and data
transfer IDs in separate clusters. The results confirm
that related CAN messages are clustered
together by unsupervised learning algorithms.
Given that J1939 CAN data has known functions,
we were able to confirm that CAN IDs with
related functions cluster together.
CONCLUSION
In this article, we have motivated the need for
vehicle security in smart cities and evaluate how
well unsupervised learning may characterize the
externally visible actions of vehicles with messages
transmitted in the CAN bus. The experimental
results show that unsupervised clusteringmethods
are able to classify, identify, and label specific CAN
IDs by their respective functions. These methods
only require timestamped CAN logs-they do not
require insider knowledge or previously reverseengineered
data. Our approach applies to all
vehicles but we could only validate the algorithm
on J1939 CAN protocol because the DBC for other
vehicles was not accessible. We could easily integrate
this approach to any vehicle that utilizes CAN
related protocols. Such low-effort capabilities,
combined with the data-rich smart city ecosystem,
may lead to greater understanding of how to secure
connected vehicles in the future.
ACKNOWLEDGMENTS
This work was supported by the National Science
Foundation under Grant CNS 1646317,
Grant CNS 1645987, and Grant CNS 2011620.
Figure 6. PCA of AH clustering.
January/February 2022
&REFERENCES
1. T. Alladi, V. Chamola, B. Sikdar, and K. R. Choo,
" Consumer IoT: Security vulnerability case studies and
solutions, " IEEEConsum. Electron. Mag., vol. 9, no. 2,
pp. 17-25, Mar. 2020.
55
IEEE Consumer Electronics Magazine - January/February 2022

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - January/February 2022
