IEEE Technology and Society Magazine - September 2023 - 61

To utilize AI in the military, the various military
departments must conduct a careful evaluation
based on their desired applications and should
weigh the advantages and disadvantages of using
AI. In addition, they should apply it in a way that
causes the least damage to human societies and the
environment.
RL and DRL algorithms for
cyber-attacks and cyber-defense
Using RL and DRL algorithms, both in cyber-attack
and cyber-defense are hampered by some
challenges. Although many RL and DRL algorithms
present superior performance, hyperparameters and
reward function design highly influence their performance.
As a result, there is a great deal of uncertainty
in training the controllers [18].
Many RL and DRL algorithms operate based on
Markov decision processes (MDP), which are fully
observable in the environment. For CPS, in particular,
the existence of an observable environment is
false due to inherent uncertainties in states. This type
of system is modeled by utilizing a partially observable
MDP (POMDP) [18].
Despite such challenges, RL and DRL algorithms
are now widely used as powerful learning techniques
in various fields, especially in cyber-attacks
and cyber-defense.
Cyber-attacks on CPS
Cyber-attacks on smart grids
Since the smart grid is a complex CPS as an electrical
network, including various operations and
energy measures, cyber-attacks on this network can
damage the entire system. Attacks must be detected
early for a smart grid to operate securely and reliably
[19]. Due to the system's vulnerability to malicious
attacks [20], we first examine these attacks on the
smart grid. Malicious attacks are divided into two
categories: physical attacks and cyber-attacks [21].
In physical attacks, the components of the power
grids are physically attacked and damaged. These
components include transformers, transmitters and
transmission lines, generators, and others. This type
of attack can be detected by the topology and data
processing modules located in the control center.
Unlike physical attacks, power grid components
are not physically attacked in cyber-attacks. Still,
the measurable data transmitted in the supervisory
September 2023
control and data acquisition (SCADA) systems are
modified and compromised. A well-planned and
sophisticated cyber-attack can manipulate the topology
and data processing modules and completely
mislead or disrupt the performance of the control
center [22].
Wang et al. [20] have proposed a coordinated
topology attack to overload a critical line by misleading
the control center. This method integrates
physical-attack and cyber-attack to perform coordinated
topology attacks using DRL algorithms.
The functions of the coordinated topology attack
are: 1) the physical attack interrupts the transmission
line; 2) it hides the cut line signal in the cyber
layer (masking a physical tripped line); and 3) it
generates a fake cut-off signal for the following
transmission line (creating a cyber tripped line).
There are two types of DRL algorithms in this
method: one to determine the attack strategy to
overload a critical transmission line, and another
one to select minimal attack resources to inject
the cyber-attack with limited attack resources.
The DRL algorithms utilized for the coordinated
topology attack are based on deep Q-learning
algorithms [20].
Cyber-attacks on software-defined
networking (SDN)
" SDN is a network architecture approach that
enables the network to be intelligently and centrally
controlled,
or programed, using software
applications " [23]. When the SDN's cyber-defense
system is carried out using RL algorithms, the
cyber-attacker tries to modify or destroy the training
process in the RL algorithm. A cyber-attacker can
perform this change and destruction in two ways:
first, forging and changing part or all of the reward
signal; second, manipulating and modifying some
states received from the environment [24]. Therefore,
when the defender agent receives the wrong
state and reward from the environment, it cannot
perform the correct and optimal action in response
to the inaccurate data. As a result, the system can
be compromised.
Unknown cyber-attacks on CPS
The major challenge that sometimes arises is the
unknown cyber-attacks that threaten any CPS. Since
the cyber-attack type is unknown, an online cyber-attack
detection method should immediately perform
61
IEEE Technology and Society Magazine - September 2023

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - September 2023
