Computational Intelligence - February 2015 - 32

Therefore, we continue with only the top learners, i.e. GPF,
MPLCS, and SBBJ. It is very interesting to observe that some
combinations of GPF and MPLCS classifiers have a great
potential, reaching values of up to 0.92.
We now study the similarities of the predictions of the five
fused models, each built with models obtained with a single
learning algorithm. In this case, we employ the probabilities
returned by the logistic regression process rather than the predictions or labels. Since these probabilities are continuous values, we can generate the 5 # 5 correlation matrix, where each
row corresponds to one of the five learners. The patterns
observed in the analysis of individuals classifiers also take place
in this case. We can see that RL-100 and RT-100 are highly
uncorrelated with the other methods. SBBJ-100 is correlated
to both GPF-100 and MPLCS-100. However, as observed in
previous results, the correlation between GPF-100 and SBBJ100 is lower, thus confirming the great potential of combining
these two learners.
VIII. Conclusion

We have introduced FCUBE, a machine learning framework
that harnesses cloud computing to solve largescale supervised
learning problems via massive ensemble learning. It exploits the
enhanced software transferability provided by virtualized cloud
resources to automate the use of learning algorithms developed
within the Evolutionary Computation community. The framework interfaces are carefully designed to enable machine learning researchers to easily import their learners and benefit from
a massive data-parallel deployment of their algorithm with
minimal overhead on their part. We refer to this concept as
Bring Your Own Learner (BYOL).
We have demonstrated the framework by integrating five
different learners and deploying them in a massive data-parallel fashion on Amazon EC2. We execute 100 runs per learner,
each with 1% of the data, in as little as one hour. The
employed algorithms are representative of evolutionary computation state-of-the-art. In particular, two of them are highly
validated algorithms provided by external collaborators. We
present results on a publicly available problem composed of
11 million exemplars based upon the Higgs dataset. The
ensemble strategies adopted in this work are promising since
we obtain a competitive performance while aiming at fast
learning. We also analyze the remarkable diversity of the classifiers trained with the different evolutionary computation
techniques. The goal of such analysis is to determine appropriate combinations of learners for future problems that the
framework could encounter.
This project aspires to a commons where new large-scale
problems can be uploaded and quickly solved thanks to a community-shared repository of learning algorithms and cloud
computing. We encourage the machine learning community to
contribute to the repository of learners and we offer the framework to all others who need a large-scale, supervised machine
learning tool.

32

IEEE ComputatIonal IntEllIgEnCE magazInE | fEbruary 2015

Acknowledgments

The authors would like to thank Dr. J. Bacardit, Dr. M. I. Heywood, and R. Smith for contributing their learning algorithms
to the FCUBE framework.
References

[1] P. Huijse, P. Estevez, P. Protopapas, J. Principe, and P. Zegers, "Computational intelligence challenges and applications on large-scale astronomical time series databases,"
IEEE Comput. Intell. Mag., vol. 9, no. 3, pp. 27-39, Aug. 2014.
[2] D. Harris, (2012, Aug. 14). How Oxdata wants to help everyone become data scientists. [Online]. Available: http://gigaom.com/2012/08/14/how-0xdata-wants-to-helpeveryone-become-data-scientists/
[3] T. Kraska, A. Talwalkar, J. C. Duchi, R. Griffith, M. J. Franklin, and M. I. Jordan,
"MLbase: A distributed machine-learning system," in Proc. 6th Biennial Conf. Innovative
Data Systems Research, 2013.
[4] Y. Low, J. Gonzalez, A. Kyrola, D. Bickson, C. Guestrin, and J. M. Hellerstein,
"Graphlab: A new framework for parallel machine learning,"in Proc. 26th Conf. Uncertainty
Artificial Intelligence, Catalina Island, CA, July 8-11, 2010.
[5] (2014). FCUBE. Project website. [Online]. Available: http://f lexgp.github.io/FCUBE
[6] J. Bacardit, P. Widera, A. Márquez-Chamorro, F. Divina, J. S. Aguilar-Ruiz, and N.
Krasnogor. (2012). Contact map prediction using a large-scale ensemble of rule sets and the
fusion of multiple predicted structural features. Bioinformatics [Online]. 28(19), pp. 2441-
2448. Available: http://bioinformatics.oxfordjournals.org/content/28/19/2441.abstract
[7] Y. Zhang and S. Bhattacharyya. (2004, June). Genetic programming in classifying
large-scale data: An ensemble method. Inf. Sci. [Online]. 163(1-3), pp. 85-101. Available:
http://dx.doi.org/10.1016/j.ins.2003.03.028
[8] N. Holden and A. A. Freitas. (2008, Oct.). Hierarchical classification of protein function with ensembles of rules and particle swarm optimisation. Soft Comput. [Online].
13(3), pp. 259-272. Available: http://dx.doi.org/10.1007/s00500-008-0321-0
[9] P. Collet, E. Lutton, M. Schoenauer, and J. Louchet, "Take it EASEA," in Parallel
Problem Solving From Nature PPSN VI (Lecture Notes in Computer Science, vol. 1917), M.
Schoenauer, K. Deb, G. Rudolph, X. Yao, E. Lutton, J. Merelo, and H.-P. Schwefel, Eds.
Berlin Heidelberg, Germany: Springer, 2000, pp. 891-901.
[10] F.-A. Fortin, F.-M. De Rainville, M.-A. Gardner, M. Parizeau, and C. Gagné,
"DEAP: Evolutionary algorithms made easy," J. Machine Learn. Res., vol. 13, pp. 2171-
2175, July 2012.
[11] T. Weise, R. Chiong, J. Lassig, K. Tang, S. Tsutsui, W. Chen, Z. Michalewicz, and X.
Yao, "Benchmarking optimization algorithms: An open source framework for the traveling salesman problem," IEEE Comput. Intell. Mag., vol. 9, no. 3, pp. 40-52, Aug. 2014.
[12] (2014). Big learning activity. in Proc. EC Big Data Big Learning workshop Held Within
GECCO Conf., [Online]. Available: http://f lexgp.github.io/EC-for-Big-Learning
[13] K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, "A fast and elitist multiobjective genetic algorithm: NSGA-II," IEEE Trans. Evol. Comput., vol. 6, no. 2, pp. 182-197, Apr. 2002.
[14] E. Vladislavleva, "Model-based problem solving through symbolic regression via pareto
genetic programming," Ph.D. dissertation, Tilburg Univ., Tilburg, The Netherlands, 2008.
[15] I. Arnaldo, K. Veeramachaneni, and U. M. O'Reilly, "Building multiclass nonlinear
classifiers with GPUs," in Proc. Big Learning Workshop NIPS: Advances Algorithms Data
Management, 2013.
[16] J. Bacardit and N. Krasnogor. (2006). Smart crossover operator with multiple parents
for a Pittsburgh learning classifier system. in Proc. 8th Annu. Conf. Genetic Evolutionary Computation [Online]. pp. 1441-1448. Available: http://doi.acm.org/10.1145/1143997.1144235
[17] M. Franco, N. Krasnogor, and J. Bacardit. (2013). GAssist vs. BioHEL: Critical assessment of two paradigms of genetics-based machine learning. in Soft Computing [Online].
17(6), pp. 953-981. Available: http://dx.doi.org/10.1007/s00500-013-1016-8
[18] J. Bacardit and X. Llorá. (2013). Large-scale data mining using genetics-based machine learning. in Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery [Online]. 3(1), pp. 37-61. Available: http://dx.doi.org/10.1002/widm.1078
[19] P. Lichodzijewski and M. I. Heywood. (2008). Managing team-based problem solving with symbiotic bid-based genetic programming. in Proc. Annu. Conf.
Genetic Evolutionary Computation [Online]. pp. 363-370. Available: http://doi.acm.
org/10.1145/1389095.1389162
[20] J. A. Doucette, A. R. McIntyre, P. Lichodzijewski, and M. I. Heywood. (2012). Symbiotic coevolutionary genetic programming: A benchmarking study under large attribute
spaces. Genetic Program. Evolvable Machines [Online]. 13(1), pp. 71-101, Available: http://
dx.doi.org/10.1007/s10710-011-9151-4
[21] A. Atwater and M. I. Heywood. (2013). Benchmarking pareto archiving heuristics in the presence of concept drift: Diversity versus age. in Proc. 15th Annu. Conf.
Genetic Evolutionary Computation [Online]. pp. 885-892. Available: http://doi.acm.
org/10.1145/2463372.2463489
[22] R. Smith and M. Heywood. (2014). SBBJ. Project website. [Online]. Available:
http://web.cs.dal.ca/ mheywood/Code/
[23] P. Baldi, P. Sadowski, and D. Whiteson. (2014, July). Searching for exotic particles
in high-energy physics with deep learning. Nature Commun. [Online]. 5(4308), Available:
http://dx.doi.org/10.1038/ncomms5308
[24] L. I. Kuncheva and C. J. Whitaker, "Measures of diversity in classifier ensembles
and their relationship with the ensemble accuracy," Machine Learn., vol. 51, no. 2, pp.
181-207, 2003.
http://www.gigaom.com/2012/08/14/how-0xdata-wants-to-help http://flexgp.github.io/FCUBE http://bioinformatics.oxfordjournals.org/content/28/19/2441.abstract http://dx.doi.org/10.1016/j.ins.2003.03.028 http://dx.doi.org/10.1007/s00500-008-0321-0 http://flexgp.github.io/EC-for-Big-Learning http://doi.acm.org/10.1145/1143997.1144235 http://dx.doi.org/10.1007/s00500-013-1016-8 http://dx.doi.org/10.1002/widm.1078 http://www.doi.acm http://http:// http://dx.doi.org/10.1007/s10710-011-9151-4 http://www.doi.acm http://web.cs.dal.ca/ http://dx.doi.org/10.1038/ncomms5308
Table of Contents for the Digital Edition of Computational Intelligence - February 2015
