IEEE Robotics & Automation Magazine - September 2015 - 146

for the RC 2-D simulation league. Our results demonstrate
that a hybrid format [25] minimizes fluctuations from true
(statistically significant) team performance rankings within
the time constraints of the RC world finals.
Our experimental analysis and many others in [10], [11],
[12], and [24] would be impossible with physical robots
alone, and have widespread applications beyond the scope of
simulated soccer matches. We encourage other researchers to
explore the potential for enriching their experimental pipelines with simulated components to minimize the experimental costs and enable others to replicate and expand upon
experimental results in a hardware-independent manner.
References
[1] J. Boedecker and M. Asada, "SimSpark-Concepts and applications in the
RoboCup 3D soccer simulation league," in Proc. Workshop SIMPAR Int. Conf.
Simulation, Modeling Programming Autonomous Robots, 2008, pp. 174-181.
[2] N. Koenig and A. Howard, "Design and use paradigms for gazebo, an
open-source multi-robot simulator," in Proc. IEEE/RSJ Int. Conf. Intelligent
Robots Systems, 2004, vol. 3, pp. 2149-2154.
[3] O. Michel, "Webots: Symbiosis between virtual and real mobile robots," in Virtual Worlds. Berlin, Germany: Springer-Verlag, 1998, pp. 254-263.
[4] S. Carpin, M. Lewis, J. Wang, S. Balakirsky, and C. Scrapper, "USARSim: A
robot simulator for research and education," in Proc. IEEE Int. Conf. Robotics
Automation, 2007, pp. 1400-1405.
[5] M. Chen, E. Foroughi, F. Heintz, Z. Huang, S. Kapetanakis, K. Kostiadis, J.
Kummeneje, I. Noda, O. Obst, P. Riley, T. Steffens, Y. Wang, and X. Yin, "RoboCup soccer server," Tech. Rep., Aug. 2002.
[6] RoboCup Technical Committee. (2013). RoboCup Soccer Simulation
League 3D competition rules and setup. [Online]. Available: http://homepages.
herts.ac.uk/sv08aav/RCSoccerSim3DRules2013.1.pdf
[7] O. Michel, Y. Bourquin, and J.-C. Baillie, "Robotstadium: Online humanoid robot soccer simulation competition," in RoboCup 2008: Robot Soccer
World Cup XII. Berlin, Germany: Springer, 2009, pp. 580-590.
[8] C. Aguero, N. Koenig, I. Chen, H. Boyer, S. Peters, J. Hsu, B. Gerkey, S.
Paepcke, J. L. Rivero, J. Manzo, E. Krotkov, and G. Pratt, "Inside the virtual
robotics challenge: Simulating real-time robotic disaster response," IEEE
Trans. Automat. Sci. Eng., vol. 12, no. 2, pp. 494-506, 2015.
[9] S. Balakirsky, R. Madhavan, and C. Scrapper, "NIST/IEEE virtual manufacturing automation competition: From earliest beginnings to future directions," in
Proc. 8th Workshop Performance Metrics Intelligent Systems, 2008, pp. 214-219.
[10] P. MacAlpine, S. Barrett, D. Urieli, V. Vu, and P. Stone, "Design and optimization of an omnidirectional humanoid walk: A winning approach at the
robocup 2011 3D simulation competition," in Proc. 26th Conf. Artificial Intelligence, 2012, pp. 1047-1053.
[11] C. Niehaus, T. Röfer, and T. Laue, "Gait optimization on a humanoid robot
using particle swarm optimization," in Proc. IEEE 2nd Workshop Humanoid
Soccer Robots, 2007, pp. 1-7.
[12] D. Budden and M. Prokopenko, "Improved particle filtering for pseudo
uniform belief distributions in robot localisation," in RoboCup 2013: Robot
Soccer World Cup XVII. Berlin, Germany: Springer-Verlag, 2013.
[13] H. Kitano, M. Asada, Y. Kuniyoshi, I. Noda, and E. Osawa, "RoboCup: The robot
world cup initiative," in Proc. 1st Int. Conf. Autonomous Agents, 1997, pp. 340-347.
[14] D. Budden and A. Mendes, "Unsupervised recognition of salient colour
for real-time image processing," in RoboCup 2013: Robot Soccer World Cup
XVII. Berlin, Germany: Springer, 2013.

146

*

IEEE ROBOTICS & AUTOMATION MAGAZINE

*

September 2015

[15] J. Fountain, J. Walker, D. Budden, A. Mendes, and S. Chalup, "Motivated reinforcement learning for improved head actuation of humanoid robots," in RoboCup 2013: Robot Soccer World Cup XVII. Berlin, Germany: Springer, 2013.
[16] M. Prokopenko, O. Obst, P. Wang, D. Budden, and O. Cliff, "Gliders2013: Tactical analysis with information dynamics," in Proc. RoboCup
Symp. Competitions: Team Description Papers, Eindhoven, The Netherlands, June 2013.
[17] M. Prokopenko, P. Wang, and O. Obst, "Gliders2014: Dynamic tactics
with voronoi diagrams," in Proc. RoboCup Symp. Competitions: Team Description Papers, Joao Pessoa, Brazil, July 2014, pp. 1-6.
[18] A. Bai, X. Chen, P. MacAlpine, D. Urieli, S. Barrett, and P. Stone, "Wrighteagle and UT Austin villa: RoboCup 2011 simulation league champions," in
RoboCup 2011: Robot Soccer World Cup XV. Berlin, Germany: Springer, 2012,
pp. 1-12.
[19] I. Noda and P. Stone, "The RoboCup soccer server and CMUnited clients:
Implemented infrastructure for MAS research," Auton. Agents Multi-Agent
Syst., vol. 7, nos. 1-2, pp. 101-120, 2003.
[20] World Bank. (2012). World development indicators 2012: GDP per capita,
PPP (current international $). [Online]. Available: http://data.worldbank.org/
data-catalog/world-development-indicators and http://data.worldbank.org/indicator/NY.GDP.PCAP.PP.CD
[21] L. Vilar, D. Ara´ujo, K. Davids, and Y. Bar-Yam, "Science of winning soccer: Emergent pattern-forming dynamics in association football," J. Syst. Sci.
Complexity, vol. 26, no. 1, pp. 73-84, 2013.
[22] D. Grabiner. (2004). The sabermetrics manifesto. [Online]. Available:
http://seanlahman.com/baseball-archive/
[23] J. Fewell, D. Armbruster, J. Ingraham, A. Petersen, and J. Waters, "Basketball teams as strategic networks," PloS one, vol. 7, no. 11, p. e47445, 2012.
[24] O. Cliff, J. Lizier, R. Wang, P. Wang, O. Obst, and M. Prokopenko, "Towards quantifying interaction networks in a football match," in RoboCup 2013:
Robot Soccer World Cup XVII. Berlin, Germany: Springer, 2013.
[25] D. Budden, P. Wang, O. Obst, and M. Prokopenko, "Simulation leagues:
Analysis of competition formats," in RoboCup 2014: Robot Soccer World Cup
XVIII. Berlin, Germany: Springer, 2014.
[26] H. Akiyama and T. Nakashima, "HELIOS Base: An open source package
for the RoboCup soccer 2D simulation," in RoboCup 2013: Robot World Cup
XVII. Berlin, Germany: Springer, 2014, pp. 528-535.
[27] D. G. Hurley, D. M. Budden, and E. J. Crampin, "Virtual reference environments: A simple way to make research reproducible," Briefings in Bioinformatics, 2014.

David M. Budden, The University of Melbourne, Parkville,
Victoria, Australia. E-mail: david.budden@unimelb.edu.au.
Peter Wang, Commonwealth Scientific and Industrial Research Organisation, Marsfield, Australia. E-mail: peter.
wang@csiro.au.
Oliver Obst, Commonwealth Scientific and Industrial Research Organisation, Marsfield, Australia. E-mail: Oliver.
Obst@csiro.au.
Mikhail Prokopenko, The University of Sydney, New South
Wales, Australia. E-mail: mikhail.prokopenko@sydney.edu.au.
http://data.worldbank.org/ http://data.worldbank.org/in http://www.seanlahman.com/baseball-archive/ http://homepages http://herts.ac.uk/sv08aav/RCSoccerSim3DRules2013.1.pdf
Table of Contents for the Digital Edition of IEEE Robotics & Automation Magazine - September 2015
