Signal Processing - November 2017 - 38

[48] V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. P. Lillicrap, T. Harley, D.
Silver, and K. Kavukcuoglu, "Asynchronous methods for deep reinforcement learning," in Proc. Int. Conf. Learning Representations, 2016.

[74] S. Singh, D. Litman, M. Kearns, and M. Walker, "Optimizing dialogue management with reinforcement learning: Experiments with the NJFun system," J. Artificial
Intell. Res., vol. 16, pp. 105-133, Feb. 2002.

[49] S. Mohamed and D. J. Rezende, "Variational information maximisation for
intrinsically motivated reinforcement learning," in Proc. Neural Information
Processing Systems, 2015, pp. 2125-2133.

[75] B. C. Stadie, S. Levine, and P. Abbeel, "Incentivizing exploration in reinforcement
learning with deep predictive models," in NIPS Workshop on Deep Reinforcement
Learning, 2015.

[50] O. Nachum, M. Norouzi, K. Xu, and D. Schuurmans. (2017). Bridging the gap
between value and policy based reinforcement learning. arXiv. [Online]. Available:
https://arxiv.org/abs/1702.08892

[76] A. L. Strehl, L. Li, E. Wiewiora, J. Langford, and M. L. Littman, "PAC model-free
reinforcement learning," in Proc. Int. Conf. Machine Learning, 2006, pp. 881-888.

[51] A. Nair, P. Srinivasan, S. Blackwell, C. Alcicek, R. Fearon, A. de Maria, V.
Panneershelvam, M. Suleyman, et al., "Massively parallel methods for deep reinforcement learning," in ICML Workshop on Deep Learning, 2015.
[52] A. Y. Ng and S. J. Russell, "Algorithms for inverse reinforcement learning," in
Proc. Int. Conf. Machine Learning, 2000, pp. 663-670.
[53] A. Y. Ng, A. Coates, M. Diel, V. Ganapathi, J. Schulte, B. Tse, E. Berger, and
E. Liang, "Autonomous inverted helicopter flight via reinforcement learning," in
Proc. Int. Symp. Experimental Robotics, 2006, pp. 363-372.
[54] B. O'Donoghue, R. Munos, K. Kavukcuoglu, and V. Mnih, "PGQ: Combining
policy gradient and Q-learning," in Proc. Int. Conf. Learning Representations,
2017.

[77] S. Sukhbaatar, A. Szlam, and R. Fergus, "Learning multiagent communication
with backpropagation," in Proc. Neural Information Processing Systems, 2016, pp.
2244-2252.
[78] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction.
Cambridge, MA: MIT Press, 1998.
[79] R. S. Sutton, D. Precup, and S. Singh, "Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning," Artificial Intell., vol. 112, no.
1-2, pp. 181-211, 1999.
[80] A. Tamar, Y. Wu, G. Thomas, S. Levine, and P. Abbeel, "Value iteration networks," in Proc. Neural Information Processing Systems, 2016, pp. 2154-2162.
[81] G. Tesauro, "Temporal difference learning and TD-gammon," Commun. ACM,
vol. 38, no. 3, pp. 58-68, 1995.

[55] J. Oh, X. Guo, H. Lee, R. L. Lewis, and S. Singh, "Action-conditional video
prediction using deep networks in Atari games," in Proc. Neural Information
Processing Systems, 2015, pp. 2863-2871.

[82] C. Tessler, S. Givony, T. Zahavy, D. J. Mankowitz, and S. Mannor, "A deep hierarchical approach to lifelong learning in Minecraft," in Proc. Association for the
Advancement Artificial Intelligence, 2017, pp. 1553-1561.

[56] I. Osband, C. Blundell, A. Pritzel, and B. van Roy, "Deep exploration via bootstrapped DQN," in Proc. Neural Information Processing Systems, 2016, pp. 4026-
4034.

[83] J. N. Tsitsiklis and B. van Roy, "Analysis of temporal-difference learning with
function approximation," in Proc. Neural Information Processing Systems, 1997, pp.
1075-1081.

[57] D. Pathak, P. Agrawal, A. A. Efros, and T. Darrell, "Curiosity-driven exploration by self-supervised prediction," in Proc. Int. Conf. Machine Learning, 2017, pp.
2778-2787.

[84] E. Tzeng, C. Devin, J. Hoffman, C. Finn, X. Peng, S. Levine, K. Saenko, and T.
Darrell, "Towards adapting deep visuomotor representations from simulated to real
environments," in Workshop Algorithmic Foundations Robotics, 2016.

[58] P. Peng, Q. Yuan, Y. Wen, Y. Yang, Z. Tang, H. Long, and J. Wang. (2017).
Multiagent bidirectionally-coordinated nets for learning to play StarCraft combat
games. arXiv. [Online]. Available: https://arxiv.org/abs/1703.10069

[85] N. Usunier, G. Synnaeve, Z. Lin, and S. Chintala, "Episodic exploration for deep
deterministic policies: An application to StarCraft micromanagement tasks," in Proc.
Int. Conf. Learning Representations, 2017.

[59] D. A. Pomerleau, "ALVINN, an autonomous land vehicle in a neural network,"
in Proc. Neural Information Processing Systems, 1989, pp. 305-313.

[86] H. van Hasselt, "Double Q-learning," in Proc. Neural Information Processing
Systems, 2010, pp. 2613-2621.

[60] D. J. Rezende, S. Mohamed, and D. Wierstra, "Stochastic backpropagation and
approximate inference in deep generative models," in Proc. Int. Conf. Machine
Learning, 2014, pp. 1278-1286.

[87] H. van Hasselt, A. Guez, and D. Silver, "Deep reinforcement learning with double
Q-learning," in Proc. Association for the Advancement of Artificial Intelligence, 2016,
pp. 2094-2100.

[61] M. Riedmiller, "Neural fitted q iteration-First experiences with a data efficient
neural reinforcement learning method," in Proc. European Conf. Machine
Learning, 2005, pp. 317-328.

[88] A. Vezhnevets, V. Mnih, S. Osindero, A. Graves, O. Vinyals, J. Agapiou, and K.
Kavukcuoglu. "Strategic attentive writer for learning macro-actions," in Proc. Neural
Information Processing Systems, 2016, pp. 3486-3494.

[62] G. A. Rummery and M. Niranjan, "On-line Q-learning using connectionist systems," Dept. Engineering, Univ. Cambridge, MA, Tech. Rep. CUED/F-INFENG/
TR 166, 1994.

[89] A. S. Vezhnevets, S. Osindero, T. Schaul, N. Heess, M. Jaderberg, D. Silver, and
K. Kavukcuoglu, "FeUdal networks for hierarchical reinforcement learning," in Proc.
Int. Conf. Machine Learning, 2017, pp. 3540-3549.

[63] A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Soyer, J. Kirkpatrick, K.
Kavukcuoglu, R. Pascanu, and R. Hadsell. (2016). Progressive neural networks.
arXiv. [Online]. Available: https://arxiv.org/abs/1606.04671

[90] N. Wahlström, T. B. Schön, and M. P. Deisenroth, "Learning deep dynamical
models from image pixels," in Proc. IFAC Symp. System Identification, 2015, pp.
1059-1064.

[64] A. A. Rusu, M. Vecerik, T. Rothörl, N. Heess, R. Pascanu, and R. Hadsell.
(2016). Sim-to-real robot learning from pixels with progressive nets. arXiv. [Online].
Available: https://arxiv.org/abs/1610.04286

[91] N. Wahlström, T. B. Schön, and M. P. Deisenroth, "From pixels to torques: policy learning with deep dynamical models," in ICML Workshop on Deep Learning,
2015.

[65] T. Salimans, J. Ho, X. Chen, and I. Sutskever. (2017). Evolution strategies as a
scalable alternative to reinforcement learning. arXiv. [Online]. Available: https://
arxiv.org/abs/1703.03864

[92] Z. Wang, N. de Freitas, and M. Lanctot, "Dueling network architectures for deep
reinforcement learning," in Proc. Int. Conf. Learning Representations, 2016.

[66] T. Schaul, D. Horgan, K. Gregor, and D. Silver, "Universal value function
approximators," in Proc. Int. Conf. Machine Learning, 2015, pp. 1312-1320.

[93] Z. Wang, V. Bapst, N. Heess, V. Mnih, R. Munos, K. Kavukcuoglu, and N. de
Freitas, "Sample efficient actor-critic with experience replay," in Proc. Int. Conf.
Learning Representations, 2017.

[67] T. Schaul, J. Quan, I. Antonoglou, and D. Silver, "Prioritized experience
replay," in Proc. Int. Conf. Learning Representations, 2016.

[94] C. J. C. H. Watkins and P. Dayan, "Q-learning," Mach. Learning, vol. 8, no. 3-4,
pp. 279-292, 1992.

[68] J. Schmidhuber, "A possibility for implementing curiosity and boredom in
model-building neural controllers," in Proc. Int. Conf. Simulation Adaptive
Behavior, 1991, pp. 222-227.

[95] M. Watter, J. Springenberg, J. Boedecker, and M. Riedmiller, "Embed to control:
A locally linear latent dynamics model for control from raw images," in Proc. Neural
Information Processing Systems, 2015, pp. 2746-2754.

[69] J. Schmidhuber and R. Huber, "Learning to generate artificial fovea trajectories
for target detection," Int. J. Neural Syst., vol. 2, no. 01n02, pp. 125-134, 1991.
[Online]. Ava ilable: http://www.worldscientif ic.com /doi /abs/10.1142/
S012906579100011X

[96] D. Wierstra, A. Förster, J. Peters, and J. Schmidhuber, "Recurrent policy gradients," Logic J. IGPL, vol. 18, no. 5, pp. 620-634, 2010.

[70] J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz, "Trust region
policy optimization," in Proc. Int. Conf. Machine Learning, 2015, pp. 1889-1897.
[71] J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel, "Highdimensional continuous control using generalized advantage estimation," in Proc.
Int. Conf. Learning Representations, 2016.
[72] D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, and M. Riedmiller,
"Deterministic policy gradient algorithms," in Proc. Int. Conf. Machine Learning,
2014, pp. 387-395.
[73] D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. van den Driessche, J.
Schrittwieser, I. Antonoglou, et al., "Mastering the game of go with deep neural networks and tree search," Nature, vol. 529, no. 7587, pp. 484-489, 2016.

38

[97] R. J. Williams, "Simple statistical gradient-following algorithms for connectionist
reinforcement learning," Mach. Learning, vol. 8, no. 3-4, pp. 229-256, 1992.
[98] M. Wulfmeier, P. Ondruska, and I. Posner, "Maximum entropy deep inverse
reinforcement learning," in NIPS Workshop on Deep Reinforcement Learning,
2015.
[99] K. Xu, J. Ba, R. Kiros, K. Cho, A. C. Courville, R. Salakhutdinov, R. S. Zemel,
and Y. Bengio, "Show, attend and tell: Neural image caption generation with visual
attention," in Proc. Int. Conf. Machine Learning, 2015, pp. 2048-2057.
[100] Y. Zhu, R. Mottaghi, E. Kolve, J. J. Lim, A. Gupta, L. Fei-Fei, and A. Farhadi,
"Target-driven visual navigation in indoor scenes using deep reinforcement learning," in
Proc. IEEE Int. Conf. Robotics and Automation, 2017, pp. 3357-3364.

SP



IEEE SIGNAL PROCESSING MAGAZINE

|

November 2017

|
https://www.arxiv.org/abs/1702.08892 https://www.arxiv.org/abs/1703.10069 https://www.arxiv.org/abs/1606.04671 https://www.arxiv.org/abs/1610.04286 http://https:// http://www.arxiv.org/abs/1703.03864
Table of Contents for the Digital Edition of Signal Processing - November 2017
