A New Q-Learning Based Spectrum Access Strategy

Abstract:

To optimize the opportunistic spectrum access strategy, a new Q-Learning based spectrum access strategy is proposed. The strategy can lead secondary user to select channels with maximum cumulative reward, and maximize secondary user throughput. From the simulation results, compared with random selection algorithm, the algorithm does not require prior knowledge or prediction models of the channel environment, yet can still select the optimal channel adaptively, improve the secondary user capability and attain to the convergence in short time.

• Periodical: Applied Mechanics and Materials (Volumes 380-384)
• Pages: 1920-1923
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.380-384.1920
• Online since: August 2013
• Authors: Kai Zhang, Ou Li, Bai Wei Yang, Yang Liu
• Keywords: Channel Selection, Cognitive Radio Network, Q-Learning, Spectrum Access

References

[1] S. Haykin, Cognitive radio: brain-empowered wireless communications, Selected Areas in Communications, IEEE Journal on, vol. 23, pp.201-220, (2005).

DOI: 10.1109/jsac.2004.839380

[2] Z. Qing and B. M. Sadler, A Survey of Dynamic Spectrum Access, Signal Processing Magazine, IEEE, vol. 24, pp.79-89, (2007).

DOI: 10.1109/msp.2007.361604

[3] Q. Zhao, L. Tong, and A. Swami, Decentralized cognitive mac for dynamic spectrum access, in New Frontiers in Dynamic Spectrum Access Networks, 2005. DySPAN 2005. 2005 First IEEE International Symposium on, 2005, pp.224-232.

DOI: 10.1109/dyspan.2005.1542638

[4] C. Yunxia, Z. Qing, and A. Swami, Joint Design and Separation Principle for Opportunistic Spectrum Access in the Presence of Sensing Errors, Information Theory, IEEE Transactions on, vol. 54, pp.2053-2071, (2008).

DOI: 10.1109/tit.2008.920248

[5] S. H. A. Ahmad, M. Y. Liu, T. Javidi, Q. Zhao, and B. Krishnamachari, Optimality of Myopic Sensing in Multichannel Opportunistic Access, Ieee Transactions on Information Theory, vol. 55, pp.4040-4050, Sep (2009).

DOI: 10.1109/tit.2009.2025561

[6] Y. X. Chen, Q. Zhao, and A. Swami, Distributed Spectrum Sensing and Access in Cognitive Radio Networks With Energy Constraint, Ieee Transactions on Signal Processing, vol. 57, pp.783-797, Feb (2009).

DOI: 10.1109/tsp.2008.2007928

[7] B. B. Wang, Z. Ji, K. J. R. Liu, and T. C. Clancy, Primary-Prioritized Markov Approach for Dynamic Spectrum Allocation, Ieee Transactions on Wireless Communications, vol. 8, pp.1854-1865, Apr (2009).

DOI: 10.1109/t-wc.2008.080031

[8] Y. H. Xu, J. L. Wang, Q. H. Wu, A. Anpalagan, and Y. D. Yao, Opportunistic Spectrum Access in Unknown Dynamic Environment: A Game-Theoretic Stochastic Learning Solution, Ieee Transactions on Wireless Communications, vol. 11, pp.1380-1391, Apr (2012).

DOI: 10.1109/twc.2012.020812.110025

[9] A. Gosavi, Reinforcement Learning: A Tutorial Survey and Recent Advances, Informs Journal on Computing, vol. 21, pp.178-192, Spr (2009).

DOI: 10.1287/ijoc.1080.0305

[10] A. Weissensteiner, A Q-Learning Approach to Derive Optimal Consumption and Investment Strategies, IEEE Transactions on Neural Networks, vol. 20, pp.1234-1243, Aug (2009).

DOI: 10.1109/tnn.2009.2020850