Deep Boltzmann Machines Aided Design Based on Genetic Algorithms

Abstract:

To resolve the problem of no guidance about how to set the values of numerical meta-parameters and difficulty to achieve optimization of Deep Boltzmann Machines, genetic algorithms are used to develop an automatic optimizing method named GA-RBMs (Genetic Algorithm-Restricted Boltzmann Machines) for this model’s aided design. Based on the Restricted Boltzmann Machines’ features and evaluation function, a genetic algorithm is designed and realizes the global search of satisfied structure. We also initialize the network’s weights to determine the number of visible units and hidden units. The experiments were conducted on MNIST digits handwritten datasets. The results proved that this optimization reduced the dimension of visible units and improved the performance of feature extracted by Deep Boltzmann Machines. The network optimized has good generalization performance and meets the demand of Deep Boltzmann Machines’ aided design.

• Periodical: Applied Mechanics and Materials (Volumes 568-570)
• Pages: 848-851
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.568-570.848
• Online since: June 2014
• Authors: Kai Liu*, Li Min Zhang, Yong Wei Sun
• Keywords: Deep Boltzmann Machines, Deep Learning, Genetic Algorithm (GA), Optimization, Restricted Boltzmann Machines

References

[1] Asja Fischer, Christian Igel, An Introduction to Restricted Boltzmann Machines. In Iberoamerican Congress on Pattern Recognition, volume 22, p.14–36, (2012).

[2] G. E. Hinton, R. Salakhutdinov, Reducing the dimensionality of data with neural networks. Science, vol. 313, p.504–507, (2006).

DOI: 10.1126/science.1127647

[3] R. Salakhutdinov, G. E. Hinton, Semantic hashing. International Journal of Approximate Reasoning, vol. 50, no. 7, p.969–978, (2009).

DOI: 10.1016/j.ijar.2008.11.006

[4] Jaehun Lee, Wooyong Chung, Eun tai Kim, A New Genetic Approach for Structure Learning of Bayesian Networks Matrix Genetic Algorithm. International Journal of Control Automation and Systems, vol. ED-2, pp.398-407, (2010).

DOI: 10.1007/s12555-010-0227-3

[5] G. E. Hinton, A practical guide to training Restricted Boltzmann Machines. In Technical report 2010-003, Machine Learning Group, University of Toronto, (2010).

[6] S. Tokdar, R. Kass, Importance sampling: a review Wiley Interdisciplinary Reviews. Computational Statistics, vol. ED-1, pp.54-60, (2010).

[7] R. Salakhutdinov, G. E. Hinton. Deep Boltzmann machines. In Proceedings of the International Conference on Artificial Intelligence and Statistics, volume 12, p.14–36, (2009).