IEEE Technology and Society Magazine - September 2023 - 63

enhance the robustness and adaptability of the
defense strategies in the face of unknown threats.
Addressing unknown cyber-attacks on CPS requires
integrating dynamic threat assessment and continuous
learning. Implementing an ensemble of RL
and DRL algorithms to collaborate and exchange
information during runtime can lead to a more
robust and adaptive cyber-defense system. By sharing
insights and experiences, these agents can collectively
learn to identify, categorize, and respond
to novel cyber-attacks in real-time. Moreover, the
ensemble approach can offer fault tolerance, as the
failure of a single algorithm would not compromise
the entire defense mechanism.
Cyber-defense of autonomous vehicle
systems:
An adversarial DRL algorithm was presented in
[26] to maximize the robustness of autonomous
vehicle dynamics control when subjected to CPS
attacks. In addition, the authors investigated state
estimation while monitoring autonomous vehicle
dynamics in the presence of CPS attacks. Based
on game theory, Ferdowsi et al. [26] analyzed how
the autonomous vehicle responds to CPS attacks
and how the attacker acts. Using inaccurate data
feeding into autonomous vehicle sensor readings,
the attacker could disrupt the optimal safe spacing
between vehicles and cause autonomous vehicle
collisions. As a defender, the autonomous vehicle
attempts to minimize spacing errors so that it is
resistant to attackers' destructive actions. Given the
possibility of manipulating data values and that the
autonomous vehicle has no idea how the attacker
plans to attack, each player learns the expected
spacing error using long short-term memory
(LSTM). An RL algorithm processes the error after
being fed by the player. The attacker's RL algorithm
then attempts to make the attack as successful as
possible, while the autonomous vehicle's RL algorithm
explores the optimal action to minimize the
spacing error. This is done with the Q-learning RL
algorithm [26].
Developing an autonomous defender agent,
learning from simulated and real-world attack scenarios,
can improve countermeasures in autonomous
vehicle systems. Utilizing RL from human
feedback (RLHF), human experts intervene to
provide feedback on the defender's actions, enabling
it to adopt strategies from human expertise.
September 2023
Evolutionary algorithms optimize the defender's
policies against diverse and rapidly evolving attacks,
enhancing system resilience.
Other applications of RL and DRL
algorithms in cyber-security
ML techniques, including RL and DRL, have been
increasingly applied to cyber-security. RL algorithms
have shown promise in intrusion detection and network
security, while DRL algorithms combine DL
and RL in complex cyber-security applications. This
section explores other applications of RL and DRL
algorithms in cyber-security:
Generating a virtualized smart city network
resource allocation that aims to assign virtual
resources to a specific user, optimally, using a
double dueling DQN RL algorithm [27]. The proposed
double dueling DQN RL algorithm can be
augmented with a multiobjective optimization
approach to enhance resource allocation in smart
city networks. Considering energy efficiency, user
fairness, and service quality, the RL model can generate
resource allocation policies that better balance
diverse objectives, making smart cities more sustainable
and equitable.
Using an A3C RL algorithm, one can create a
mobile edge caching to maximize offloading traffic
[28]. In addition to traffic maximization, the A3C
RL algorithm can be extended to prioritize content
caching based on temporal access patterns. By anticipating
the popularity of content over time, the RL
model can proactively cache frequently requested
data, reducing latency and enhancing the overall
quality of experience for users in mobile edge caching
scenarios.
Robustness-guided falsification of CPS can be
done to find false inputs in these systems by integrating
double DQN, and A3C RL algorithms [29]. To
improve the robustness of falsification detection in
CPS, the RL model can be enriched with an ensemble
learning approach. The system can achieve more
comprehensive and reliable falsification detection
results by combining multiple RL algorithms and
complementary techniques, such as rule-based
methods or physics-based models.
system against
Enhancing the robustness of the autonomous
adversarial
attacks
to recognize
corrupted measurements and decrease the effects
of malicious errors using the trust region policy
optimization (TRPO) RL algorithm [30]. Beyond
63
IEEE Technology and Society Magazine - September 2023

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - September 2023
