Multi-User Opportunistic Spectrum Access Using Reinforcement Learning

Abstract:

This paper studies the channel exploration problem for the distributed opportunistic spectrum access (D-OSA) system, where multiple secondary users (SUs) sequentially sense multiple licensed channels and utilize one of idle channel. However, channel sensing order can affect the system performance seriously. When using a better sensing order, the SU can find faster a free channel with high quality and the less collisions among SUs can happen. In this paper, we propose a mechanism using reinforcement learning to find dynamically out a sensing order for improving the system performance. In the proposed mechanism, the interactions among SUs are considered. Simulation results are provided to show the effectiveness of the proposed mechanism and the significant improvement of the system performance.

• Periodical: Advanced Materials Research (Volumes 926-930)
• Pages: 2357-2361
• DOI: https://doi.org/10.4028/www.scientific.net/AMR.926-930.2357
• Online since: May 2014
• Authors: Cheng Long Xu*, Yun Peng Cheng, Yang Chen, Cheng Meng Ren, Heng Yang
• Keywords: Multiple Users, Optimal Stopping Problem (OSP), Reinforcement Learning

References

[1] S. Haykin, Cognitive radio: brain-empowered wireless communications, IEEE Journal on Selected Areas in Communications, vol. 23, no. 2, pp.201-220, Feb. (2005).

DOI: 10.1109/jsac.2004.839380

[2] Y. Xu, A. Anpalagan and L. Shen, et al., Decision-theoretic opportunistic spectrum access: strategies, challenges and solutions, IEEE Communication Survey & Tutorial, vol. 15, no. 4, pp.1689-1713, (2013).

DOI: 10.1109/surv.2013.030713.00189

[3] A. Sabharwal, A. Khoshnevis, and E. Knightly, Opportunistic spectrum usage: Bounds and a multi-band CSMA/CA protocol, IEEE/ACM Transactions on Networking, vol. 15, no. 3, pp.533-545, Jun. (2007).

DOI: 10.1109/tnet.2007.893230

[4] H. Jiang, L. Lai, R. Fan, and H. Poor, Optimal selection of channel sensing order in cognitive radio, IEEE Transactions on Wireless Communications, vol. 8, no. 1, pp.297-307, Jan. (2009).

DOI: 10.1109/t-wc.2009.071363

[5] A. Mendes, C. Augusto and M. Silva, et al., Channel sensing order for cognitive radio networks using reinforcement learning, in 36th Annual IEEE Conference on Local Computer Networks (LCN), pp.546-553, 2011, Bonn.

DOI: 10.1109/lcn.2011.6115516

[6] S. Kim and G. Giannakis, Sequential and cooperative sensing for multichannel cognitive radios, IEEE Transactions on Signal Processing, vol. 58, no. 8, pp.4239-4253, Aug. (2010).

DOI: 10.1109/tsp.2010.2049106

[7] T. Ferguson, Optimal Stopping and Applications. [Online] Available: http: /www. math. ucla. edu/ tom/ Stopping/ Contents. html.

[8] R. Sutton and A. Barto, Reinforcement learning: A introduction, MIT Press, (1998).