Approximate Dynamic Programming in the Sensor-Based Navigation of the Wheeled Mobile Robot


Article Preview

The article presents a new approach to the sensor-based navigation of wheeled mobile robot Pioneer 2-DX in the unknown 2-D environment with static obstacles. The navigation task has been developed using a discrete hierarchical control system with a path planning layer and a tracking control layer designed using approximate dynamic programming algorithms. The navigator realises a behavioural control approach to the path planning process using the adaptive coordination of two simple behaviours: “goal-seeking” and “obstacle avoiding”. The main part of the navigator is the Action-Dependant Heuristic Dynamic Programming structure realised in a form of the actor and critic neural networks. To avoid the time consuming trial and error learning, additional proportional controllers generating signals that prompt the direction of the sub-optimal control law seeking process at the beginning of the NNs adaptation process are arranged in the navigator. The tracking control layer is composed of a PD controller, the Dual Heuristic Dynamic Programming algorithm and a supervisory term. It generates control signal for DC motors of the robot. The performance of the proposed discrete control system was verified by a series of experiments conducted using wheeled mobile robot Pioneer 2-DX equipped with one laser and eight ultrasonic range finders that provide object detection.



Solid State Phenomena (Volumes 220-221)

Edited by:

Algirdas V. Valiulis, Olegas Černašėjus and Vadim Mokšin




Z. Hendzel and M. Szuster, "Approximate Dynamic Programming in the Sensor-Based Navigation of the Wheeled Mobile Robot", Solid State Phenomena, Vols. 220-221, pp. 60-66, 2015

Online since:

January 2015




* - Corresponding Author

[1] H. Maaref, C. Barret, Sensor- based navigation of a mobile robot in an indoor environment Robot. Auton. Syst. 38 (2002) 1–18.

[2] R.C. Arkin, Behaviour-Based Robotics, MIT Press, Cambridge, (1998).

[3] J. Millan, Reinforcement learning of goal-directed obstacle-avoiding reaction strategies in an autonomous mobile robot, Robot. Auton. Syst. 15 (1995) 275–299.


[4] Z. Hendzel, M. Szuster, Neural Dynamic Programming in Reactive Navigation of Wheeled Mobile Robot, Lecture Notes in Comput. Sci. 7268 (2012) 450–457.


[5] M. Giergiel, Z. Hendzel, W. Zylski, Modeling and control of wheeled mobile robots, PWN, Warsaw, 2002, (in Polish).

[6] Z. Hendzel, M. Szuster, Discrete Action Dependant Heuristic Dynamic Programming in Wheeled Mobile Robot Control, Solid State Phenom. 164 (2010) 419–424.


[7] Z. Hendzel, M. Szuster, Discrete model-based adaptive critic designs in wheeled mobile robot control, Lecture Notes in Comput. Sci. 6114 (2010) 264–271.


[8] D.V. Prokhorov, D.C. Wunch, Adaptive critic designs, IEEE Trans. Neural Networks 8 (1997) 997–1007.


[9] W.B. Powell, Approximate Dynamic Programming: Solving the Curses of Dimensionality, Wiley Interscience, Princeton, (2012).