IEEE Electrification - March 2021 - 52

For Further Reading
F. Lian, A. Chakrabortty, and A. Duel-Hallen, " Game-theoretic
multi-agent control and network cost allocation under communication constraints, " IEEE J. Sel. Areas Commun., vol. 35, no.
2, pp. 330-340, 2017. doi: 10.1109/JSAC.2017.2659338.
F. Lin, M. Fardad, and M. R. Jovanovic, " Design of optimal
sparse feedback gains via the alternating direction method of
multipliers, " IEEE Trans. Autom. Control, vol. 58, no. 9, pp. 2426-
2431, 2013. doi: 10.1109/TAC.2013.2257618.
J. H. Chow, Ed., Power System Coherency and Model Reduction.
New York: Springer-Verlag, 2013.
S. Chandra, D. F. Gayme, and A. Chakrabortty , " Time-scale
modeling of wind-integrated power systems, " IEEE Trans.
Power Syst., vol. 31, no. 6, pp. 4712-4721, 2016. doi: 10.1109/
TPWRS.2016.2521403.
N. Xue and A. Chakrabortty, " Control inversion: A clustering-based method for distributed wide-area control of power
systems, " IEEE Trans. Control Netw. Syst., vol. 6, no. 3, pp. 937-
949, 2019. doi: 10.1109/TCNS.2018.2888997.
D. Tsubakino, T. Yoshioka, and S. Hara, " An algebraic
approach to hierarchical LQR synthesis for large-scale
dynamical systems, " in Proc. 9th Asian Control Conf., 2013,
pp. 1-6. doi: 10.1109/ASCC.2013.6606171.
T. Sadamoto, A. Chakrabortty, T. Ishizaki, and J. I. Imura,
" Retrofit control of wind-integrated power systems, " IEEE
Trans. Power Syst., vol. 33, no. 3, pp. 2804-2815, 2018. doi:
10.1109/TPWRS.2017.2750411.
G. Foderaro, S. Ferrari, and T. A. Wettergren, " Distributed
optimal control for multi-agent trajectory optimization, "
Automatica, vol. 50, no. 1, pp. 149-154, 2014. doi: 10.1016/j.automatica.2013.09.014.
F. L. Lewis and D. Vrabie, " Reinforcement learning and
adaptive dynamic programming for feedback control, " IEEE
Circuits Syst. Mag., vol. 9, no. 3, pp. 32-50, 2009. doi: 10.1109/
MCAS.2009.933854.
K. G. Vamvoudakis and F. L. Lewis, " Online actor-critic
algorithm to solve the continuous-time infinite-horizon optimal control problem, " Automatica, vol. 46, no. 5, pp. 878-888,
2010. doi: 10.1016/j.automatica.2010.02.018.
Y. Jiang and Z. P. Jiang, Robust Adaptive Dynamic Programming. Piscataway, NJ: IEEE Press, 2017.
K. S. Narendra and K. Parthasarathy, " Identification and
control of dynamical systems using neural networks, " IEEE
Trans. Neural Netw., vol. 1, no. 1, pp. 4-27, 1990. doi:
10.1109/72.80202.
K. Narendra and A. Annaswamy, " A new adaptive law for
robust adaptation without persistent excitation, " IEEE Trans.
Autom. Control, vol. 32, no. 2, pp. 134-145, 1987. doi: 10.1109/
TAC.1987.1104543.
M. Vidyasagar and R. L. Karandikar, " A learning theory
approach to system identification and stochastic adaptive
control, " in Probabilistic and Randomized Methods for Design
under Uncertainty, G. Calafiore and F. Dabbene, Eds. London:
Springer-Verlag, 2006, pp. 265-302.
R. S. Sutton and A. G. Barto, Reinforcement Learning: An
Introduction. Cambridge, MA: MIT Press, 1998.
F. L. Lewis, D. Vrabie, and K. G. Vamvoudakis, " Reinforcement learning and feedback control: Using natural decision
methods to design optimal adaptive controllers, " IEEE Control
Syst. Mag., vol. 32, no. 6, pp. 76-105, 2012. doi: 10.1109/
MCS.2012.2214134.
M. I. Abouheaf, F. L. Lewis, K. G. Vamvoudakis, S. Haesaert,
and R. Babuska, " Multi-agent discrete-time graphical games and
reinforcement learning solutions, " Automatica, vol. 50, no. 12, pp.
3038-3053, 2014. doi: 10.1016/j.automatica.2014.10.047.
K. G. Vamvoudakis, " Q-learning for continuous-time linear systems: A model free infinite horizon optimal control

52

I E E E E l e c t r i f i cati o n M agaz ine / MARCH 2021

approach, " Syst. Control Lett., vol. 100, pp. 14-20, Feb. 2017. doi:
10.1016/j.sysconle.2016.12.003.
W. B. Powell, Approximate Dynamic Programming: Solving the
Curses of Dimensionality. Hoboken, NJ: Wiley, 2007.
P. Mehta and S. Meyn, " Q-learning and Pontryagin'S minimum principle, " in Proc. IEEE Conf. Decision Control, 2009, pp.
3598-3605. doi: 10.1109/CDC.2009.5399753.
P. P. Khargonekar and M. A. Dahleh, " Advancing systems
and control research in the era of ML and AI, " Annu. Rev. Control, vol. 45, pp. 1-4, 2018. doi: 10.1016/j.arcontrol.2018.04.001.
B. Recht, " A tour of reinforcement learning: The view from
continuous control, " 2018, arXiv:1806.09460.
J. Cordova, R. Arghandeh, Y. Zhou, S. Wesolowski, W. Wu,
and S. Matthias, " Shape-based data analysis for event classification in power systems, " IEEE PowerTech, 2017, pp. 1-6, doi:
10.1109/PTC.2017.7980950.
T. K. Das, G. K. Venayagamoorthy, and U. O. Aliyu, " Bioinspired algorithms for the design of multiple optimal power
system stabilizers: SPPSO and BFA, " IEEE Trans. Ind. Appl., vol. 44,
no. 5, pp. 1445-1457, 2008. doi: 10.1109/TIA.2008.2002171.
V. Kekatos, G. B. Giannakis, and R. Baldick, " Online energy
price matrix factorization for power grid topology tracking, "
IEEE Trans. Smart Grid, vol. 7, no. 3, pp. 1239-1248, 2016. doi:
10.1109/TSG.2015.2469098.
T. Hong, J. Wilson, and J. Xie, " Long-term probabilistic load
forecasting and normalization with hourly information, " IEEE
Trans. Smart Grid, vol. 5, no. 1, pp. 456-462, 2014. doi: 10.1109/
TSG.2013.2274373.
L. Xiao, X. Xiao, C. Dai, M. Pengy, L. Wang, and H. Vincent
Poor, " Reinforcement learning-based energy trading for
microgrids, " 2018. [Online]. Available: https://arxiv.org/
abs/1801.06285
T. Sadamoto, A. Chakrabortty, T. Ishizaki, and J. Imura,
" Dynamic modeling, stability, and control of power systems
with distributed energy resources: Handling faults using two
control methods in tandem, " IEEE Control Syst. Mag., vol. 39,
no. 2, pp. 34-65, Apr. 2019. doi: 10.1109/MCS.2018.2888680.
S. Mukherjee, H. Bai, and A. Chakrabortty, " On modelfree reinforcement learning of reduced-order optimal control for singularly perturbed systems, " in Proc. IEEE Conf. Decision Control, 2018, pp. 5288-5293. doi: 10.1109/CDC.2018
.8619022.
N. Wu, Y. Bi, N. Michael, A. Tang, J. Doyle, and N. Matni,
" HFTraC: High-frequency traffic control, " ACM SIGMETRICS
Performance Eval. Rev., vol. 45, no. 1, pp. 43-44, 2017. doi:
10.1145/3143314.3078557.
D. Soudbakhsh, A. Chakrabortty, and A. M. Annaswamy, " A
delay-aware cyber-physical architecture for wide-area control
of power systems, " IFAC Control Eng. Pract., vol. 60, pp. 171-182,
Mar. 2017. doi: 10.1016/j.conengprac.2016.12.012.
M. Gol and A. Abur, " Observability and criticality analyses
for power systems measured by phasor measurements, " IEEE
Trans. Power Syst., vol. 28, no. 3, pp. 3319-3326, 2013. doi:
10.1109/TPWRS.2012.2236367.
A. K. Singh and B. C. Pal, " Decentralized dynamic state estimation in power systems using unscented transformation, "
IEEE Trans. Power Syst., vol. 29, no. 2, pp. 794-804, 2014. doi:
10.1109/TPWRS.2013.2281323.

Biography
Aranya Chakrabortty (achakra2@ncsu.edu) is with North
Carolina State University, Raleigh, North Carolina, 27606,
USA, and the National Science Foundation, Alexandria,
Virginia, 22314, USA.
https://arxiv.org/abs/1801.06285 https://arxiv.org/abs/1801.06285
IEEE Electrification - March 2021

Table of Contents for the Digital Edition of IEEE Electrification - March 2021
