A Neural-Q-Learning Based Approach for Delay Tolerant Underwater Acoustic Wireless Sensor Networks

Abstract:

Delay tolerance is a major design concern for supporting underwater acoustic wireless sensor networks (UA-WSNs) to carry out tasks in harsh subsea environments. Due to the great difference between the underwater acoustic channel and the radio frequency channel, most of the existing routing protocols developed for terrestrial wireless sensor networks perform poorly in underwater acoustic communications. In this work, we present a Neural-Q-Learning algorithm based delay tolerant protocol for UA-WSNs. Due to the advantages of the artificial neural network along with the Q-Learning algorithm, the ferry node is capable of determining an optimal route in a two-dimensional continuous space to relay packets effectively and efficiently between sensors. Simulation results show that the delivery delay and delivery cost of the network significantly decrease by maximizing the meeting probability between the ferry node and sensors.

• Periodical: Applied Mechanics and Materials (Volumes 548-549)
• Pages: 1530-1535
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.548-549.1530
• Online since: April 2014
• Authors: Lin Zou*, De Feng Huang, Roberto Togneri
• Keywords: Acoustic, Artificial Neural Network (ANN), Neural-Q-Learning, Underwater, Wireless Sensor Network (WSN)

References

[1] T. Hu, Y. Fei, QELAR: A Machine-Learning-Based Adaptive Routing Protocol for Energy-Efficient and Lifetime-Extended Underwater Sensor Networks, IEEE Transactions on Mobile Computing, June 2010, vol. 9, no. 6, pp.796-809.

DOI: 10.1109/tmc.2010.28

[2] S. Climent, JV Capella, N. Meratnia, JJ Serrano, Underwater Sensor Networks: A New Energy Efficient and Robust Architecture. Sensors. 2012; 12(1): 704-731.

DOI: 10.3390/s120100704

[3] T. Hu, Y. Fei, An Adaptive and Energy-efficient Routing Protocol Based on Machine Learning for Underwater Delay Tolerant Networks, Modeling, Analysis & Simulation of Computer and Telecommunication Systems (MASCOTS), 2010 IEEE International Symposium on , vol., no., p.381, 384, 17-19 Aug. (2010).

DOI: 10.1109/mascots.2010.45

[4] C. Gaskett, D. Wettergreen and A. Zelinsky, Q-Learning in Continuous State and Action Spaces, Lecture Notes in Computer Science, Vol 1747/1999, pp.417-428, (1999).

DOI: 10.1007/3-540-46695-9_35

[5] Anderson, J. A., 1995, Introduction to Neural Networks (Cambridge, MA: MIT Press).

[6] Limei Zhou; Liwei Zhang, A Log-Sigmoid Lagrangian Neural Network for Solving Nonlinear Programming, Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing, 2007. SNPD 2007. Eighth ACIS International Conference on , vol. 3, no., p.427, 431, July 30 2007-Aug. 1 (2007).

DOI: 10.1109/snpd.2007.15

[7] D. Levin, The approximation power of moving Least-Squares, Mathematics of Computation, vol. 67, p.1517–1531, (1998).

DOI: 10.1090/s0025-5718-98-00974-0

[8] J.G. Proakis, E.M. Sozer, J.A. Rice, M. Stojanovic, Shallow water acoustic networks, IEEE Communications Magazine 39 (11) (2001) 114–119.

DOI: 10.1109/35.965368

[9] Rumelhart, David E.; Hinton, Geoffrey E., Williams, Ronald J. (8 October 1986). Learning representations by back-propagating errors,. Nature 323 (6088): 533–536.

DOI: 10.1038/323533a0

[10] J. G. Bellingham, Small, High Performance Autonomous Underwater Vehicles, Massachusetts Institute of Technology, Technical Report 01-04, (1997).