IEEE Technology and Society Magazine - September 2023 - 62

Feature
an optimal and effective defense strategy according
to existing conditions. It is preferable to use learning-based
methods (e.g., RL and DRL algorithms
[25]) against these attacks to learn the existing conditions
in real-time and defend online.
Cyber-defense of CPS
For the cyber-defense of CPS, adopting a multilayered
defense architecture can be advantageous.
Instead of relying solely on RL or DRL algorithms
at the application level, combining rule-based
approaches, anomaly detection systems, and
RL-based decision-making can create a synergistic
defense strategy. The rule-based layer can respond
quickly and mitigate known attacks, while the RL
and DRL algorithms can handle more complex
and evolving threats. This multilayered approach
would ensure a more comprehensive and adaptable
defense against various cyber-attacks.
Cyber-defense of smart grids
Considering the problems for smart grids due to
issues related to cyber-attacks raised earlier, Kurt
et al. [19] have formulated an online cyber-attack
detection problem as a POMDP. They have provided
a robust online detection algorithm based
on model-free RL for detecting cyber-attacks that
targets a smart grid. The RL cyber-attack detection
algorithm incorporates two stages: learning
and online detection. In the learning stage, the
defender is trained by the state action reward state
action (SARSA) algorithm (a model-free RL algorithm)
and learns a Q-table. In the online detection
stage, the learned Q-table in the learning
stage is utilized to select the action with the lowest
expected future cost. Since the proposed algorithm
by Kurt et al. [19] is general, it can detect
known and unknown attacks.
In the context of cyber-attacks and -defense on
smart grids, ensuring sensor data integrity is vital
for accurate decision-making and attack detection.
We propose novel methods with a trust-based
mechanism to authenticate sensor data before
using it in RL or DRL algorithms. This mechanism
can employ cryptographic techniques like digital
signatures or secure hardware modules for data
origin and integrity validation. Integrating this
trust-based data verification enhances the smart
grid's cyber-defense system against data manipulation
attacks.
62
Cyber-defense of SDN
An RL method for autonomous cyber-defense
against attackers who intend to infiltrate and propagate
throughout the entire SDN and compromise
critical network servers was proposed in [24]. This
method trains two diverse RL agents-one agent is
based on a double DQN RL algorithm to isolate the
agents and preserve them from attacks. The second
agent is the asynchronous advantage actor-critic
(A3C) RL algorithm to keep nodes uncompromised
and reachable from the critical server. This method
allows RL agents to perform suboptimal actions
despite attacks on RL algorithms.
A proactive approach can be adopted to councyber-attacks
on SDN more effectively.
Instead of relying on reactive measures to detect
and respond to attacks during the RL training
process, the defense agents can be trained using
historical attack data and simulated adversarial
attacks. This approach could encourage the
RL agent to learn preemptive defense strategies,
allowing it to anticipate potential attacks and minimize
their impact, even when adversaries may
manipulate the reward signals.
teract
Cyber-defense of CPS
An online cyber-defense to combat unknown
cyber-attacks in a CPS was developed in [25]. First,
the cyber-state dynamics was created to assess the
effects of the current cyber-attack and the defense
strategy efficiently and dynamically in real-time.
Since cyber-attack is unknown and there is insufficient
information about the cyber-state dynamics
to provide an optimal cyber-defense, an actor-critic
neural network (NN) architecture was proposed to
learn the optimal online cyber-defense strategy effectively.
Finally, a novel actor-critic DRL algorithm was
implemented to improve the proposed method [25].
One of the significant challenges in combating
cyber-attacks on CPS is the unpredictability
of unknown cyber threats. Unlike known attacks,
which can be addressed with predesigned defense
strategies, unknown cyber-attacks demand realtime
detection and effective defense mechanisms.
We envision generative adversarial networks
(GANs) or variational autoencoders (VAEs) to create
synthetic attack scenarios that can be used to
augment training data for RL-based defense systems.
By exposing the defense agents to diverse
attack scenarios, these synthetic methods could
IEEE Technology and Society Magazine
IEEE Technology and Society Magazine - September 2023

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - September 2023
