A Metropolis Criterion Based Fuzzy Q-Learning Flow Controller for High-Speed Networks

Abstract:

For the congestion problems in high-speed networks, a Metropolis criterion based fuzzy Q-learning flow controller is proposed. Because of the uncertainties and highly time-varying, it is not easy to accurately obtain the complete information. In this case, the Q-learning, which is independent of mathematic model, and prior-knowledge, has good performance. The fuzzy inference and Metropolis criterion are introduced in order to facilitate generalization in large state space and balance exploration and exploitation in action selection individually. Simulation results show that the controller can learn to take the best action to regulate source flow with the features of high throughput and low packet loss ratio, and can avoid the occurrence of congestion effectively.

• Periodical: Applied Mechanics and Materials (Volumes 241-244)
• Pages: 2327-2330
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.241-244.2327
• Online since: December 2012
• Authors: Wen Wei Liu, Xin Li, Xiao Ning Qin, Dan Yu
• Keywords: Flow Controller, Fuzzy Q-Learning, High-Speed Networks, Metropolis Criterion

References

[1] M. Lestas, A. Pitsillides, P. Ioannou, and G. Hadjipollas, Adaptive congestion protocol: a congestion control protocol with learning capability, Computer Networks, International Journal of Computer and Telecommunications Networking, vol. 51, no. 13 (2007). pp.3773-3798.

DOI: 10.1016/j.comnet.2007.04.002

[2] R. S. Sutton and A. G. Barto, Reinforcement Learning an Introduction. Cambridge, MA.: MIT Press (1998).

[3] M. C. Hsiao, S. W. Tan, K. S. Hwang, and C. S. Wu, A reinforcement learning approach to congestion control of high-speed multimedia networks, Cybernetics and Systems, vol. 36, no. 2 (2005), pp.181-202.

DOI: 10.1080/01969720590897224

[4] X. Li, X. J. Shen, Y. W. Jing, and S. Y. Zhang, Simulated Annealing-Reinforcement Learning Algorithm for ABR Traffic Control of ATM Networks, Proc. of the 46th IEEE Conf. on Decision and Control, (2007), pp.5716-5721.

DOI: 10.1109/cdc.2007.4434121

[5] M. L. Littman, Value-function reinforcement learning in Markov games, Journal of Cognitive System Research, vol. 2, no. 1 (2001), pp.55-66.

DOI: 10.1016/s1389-0417(01)00015-8

[6] D. B. Gu, and E. F. Yang, A policy gradient reinforcement learning algorithm with fuzzy function approximation, IEEE International Conf. on Robotics and Biomimetics, (2004), pp.934-940.

DOI: 10.1109/robio.2004.1521910

[7] M. Z. Guo, Y. Liu, and J. Malec, A new Q-learning algorithm based on the metropolis criterion, IEEE Transactions on System, Man, and Cybernetics-Part B: Cybernetics, vol. 34, no. 5, (2004), pp.2140-2143.

DOI: 10.1109/tsmcb.2004.832154