Pricing Scheme Based Nash Q-Learning Flow Control for Multi-User Network

Abstract:

For the congestion problems with multi-user existing in high-speed networks, a pricing scheme based Nash Q-learning flow controller is proposed. It considers a network with a single service provider, and some non-cooperative users. The pricing scheme is introduced to the design of the reward function in the learning process of Q-learning. Because of the uncertainties and highly time-varying, it is not easy to accurately obtain the complete information for high-speed networks. The Nash Q-learning, which is independent of mathematic model, shows particular superiority. It obtains the Nash Q-values through trial-and-error and interaction with the environment to improve its behavior policy. By means of learning process, the proposed controller can learn to take the best actions to regulate source flow with the features of high quality of service. Simulation results show that the proposed controller can promote the performance of the networks and avoid the occurrence of congestion effectively.

• Periodical: Key Engineering Materials (Volumes 467-469)
• Pages: 847-852
• DOI: https://doi.org/10.4028/www.scientific.net/KEM.467-469.847
• Online since: February 2011
• Authors: Xin Li, Hai Bin Yu
• Keywords: Multi-User, Nash Q-Learning, Pricing Scheme

References

[1] R. G. Cheng, C. J. Chang and L. F. Lin: A QoS-provisioning Neural Fuzzy Connection Admission Controller for Multimedia High-speed Networks. IEEE/ACM Transactions on Networking, vol. 7, no. 1 (1999), pp.111-121.

DOI: 10.1109/90.759332

[2] M. Lestas, A. Pitsillides, P. Ioannou, and G. Hadjipollas: Adaptive Congestion Protocol: a Congestion Control Protocol with Learning Capability. Computer Networks, vol. 51, no. 13 (2007), pp.3773-3798.

DOI: 10.1016/j.comnet.2007.04.002

[3] R. S. Sutton and A. G. Barto: Reinforcement Learning an Introduction. (MIT Press, Cambridge, 1998).

[4] A. Chatovich, S. Okug, and G. Dundar: Hierarchical Neuro-fuzzy Call Admission Controller for ATM Networks. Computer Communications, vol. 24, no. 11 (2001), pp.1031-1044.

DOI: 10.1016/s0140-3664(00)00331-5

[5] M. C. Hsiao, S. W. Tan, K. S. Hwang, and C. S. Wu: A Reinforcement Learning Approach to Congestion Control of High-speed Multimedia Networks. Cybernetics and Systems, vol. 36, no. 2 (2003), pp.181-202.

DOI: 10.1080/01969720590897224

[6] F. P. KELLY, A. MAULLOO and D. TAN: Rate Control in Communication Networks: Shadow Prices, Proportional Fairness andStability. Journal of the Operations Research Society, vol. 49 (1998), pp.237-252.

DOI: 10.1038/sj.jors.2600523

[7] J. Hou, J. Yang, and S. Papavassiliou: Integration of Pricing with Call Admission Control to Meet QoS Requirements in Cellular Networks. IEEE Transactions on Parallel and Distributed System, vol. 13, no. 9 (2002), pp.898-910.

DOI: 10.1109/tpds.2002.1036064

[8] Basar T, Srikant R. Revenue Maximizing Pricing and Capacity Expansion in a Many-Users Regime. Proceedings of the IEEE INFOCOM, New York, pp.321-329 (2002).

DOI: 10.1109/infcom.2002.1019271