Mobile Robot Navigation Using ARTQL Algorithm with Novelty Driven Mechanism

Abstract:

Q-learning algorithm is usually used for traditional mobile robot navigation. The traditional Q-learning methods have the problem of dimension disaster, which may be produced by applying Q-learning to intelligent system of continuous state-space. Besides, the learning activity and efficiency are low. In order to solve these problems, a new method called ARTQL is proposed, which combined ART2 network with the traditional Q-learning algorithm. Then, a learning mechanism called novelty driven is proposed to lead the ARTQL algorithm to learn more actively and efficiently. Through the ARTQL with novelty driven mechanism algorithm, Q-learning Agent in view of the duty which needs to complete to learn an appropriate incremental clustering of state-space model, so Agent can carry out decision-making and a two-tiers online learning of state-space model cluster in unknown environment, without any priori knowledge, through interaction with the environment unceasingly alternately to improve the control strategies, increase the learning accuracy, activity and efficiency. Finally through the mobile robot navigation simulation experiments, we show that, using the proposed algorithm, mobile robot can improve its navigation performance continuously by interactive learning with the environment with high autonomous.

• Periodical: Applied Mechanics and Materials (Volumes 380-384)
• Pages: 1117-1120
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.380-384.1117
• Online since: August 2013
• Authors: Qi Xiang Hu, Xin Yu Qu
• Keywords: ARTQL, Incremental Learning, Mobile Robot Navigation, Novelty Driven

References

[1] Tom M Mitchell, Machine learning, Beijing, China: Machine Press, 2004, pp.263-280.

[2] Xiao N F, Nahavandi S, A reinforcement learning approach for robot control in an unknown environment, Proc of the IEEE International Conference on Industrial Technology. Bangkok, Thailand, pp.1096-1099, (2002).

DOI: 10.1109/icit.2002.1189324

[3] Yichi Wang, John M Usher, Application of reinforcement learning for Agent-based production scheduling, Engineering Application of Artificial Intelligence, vol. 18, pp.73-82, (2005).

DOI: 10.1016/j.engappai.2004.08.018

[4] Gao Yang, The Research On Reinforcement Learning. Machine Learning and application, Beijing University Press, China, (2006).

[5] Watkins C, Dayan P, Q-Learning, Machine Learning, vol. 8, pp.279-292, (1992).

[6] Singh S, Jaakkola T, Jordan M I, Reinforcement learning with soft state aggregation. In: Tesauro G, Touretzky D, eds. Advances in Neural Information Processing Systems. 7. Morgan Kaufmann/MIT Press, pp.361-368, (1995).

[7] Lau H Y K, Mak K L, Lee I S K, Adaptive vector quantization for reinforcement learning. In: Proceedings o f the l5th World Congress of International Federation of Automatic Control. Barcelona, Spain, pp.21-26, (2002).

[8] Davies S, Multidimensional triangulation and interpolation for reinforcement learning. In: Michael C Mozer, Michael I Jordan, Thomas Petsche, eds. Advance in Neural Information Processing Systems 9. Cambridge, MA: MIT Press, pp.1005-1010, (1997).

[9] B. Karthikeyan, S. Gopal and S. Venkatesh, ART-2: An unsupervised neural network for PD pattern recognition and classification, Expert Syst. Appl, vol. 31, no. 2, p.345–350, (2006).

DOI: 10.1016/j.eswa.2005.09.029

[10] Han Liqun, The neural network theory, design and application, Beijing. Chemical Industry Press , (2007).

[11] G. A. Carpenter and S. Grossberg, ART2: Stable self-organization of stable category recognition codes for analogue input patterns, Applied Optics, vol. 26, p.4919–4930, (1987).

DOI: 10.1364/ao.26.004919

[12] Jianhong Luo, Dezhao Chen, An Enhanced ART2 Neural Network for Clustering Analysis, Proceedings of the First International Workshop on Knowledge Discovery and Data Mining, pp.81-85, (2008).

DOI: 10.1109/wkdd.2008.117

[13] X.D. Qian, Z.O. Wang, Y. Wang, A method of data clustering based on improved algorithm of ART2, in: Proceedings of the Fourth International Conference on Machine Learning and Cybernetics, p.2021–2026, (2005).

DOI: 10.1109/icmlc.2005.1527277

[14] P. -Y. Oudeyer, F. Kaplan, and V. Hafner. Intrinsic motivation systems for autonomous mental development,. IEEE Transactions on Evolutionary Computation, 11, (2007).

DOI: 10.1109/tevc.2006.890271