This research presents a dynamic multi-robots formation navigation algorithm named multi-EKF localization algorithm, which derived from single EKF. In this algorithm, all formation members are treated as landmarks with known association. When the formation is running, the traditional EKF is used for individual robot to get its own localization, and the proposed multi-EKF is used to get the position of the whole formation. By calculating the joint probability distribution, the mean and covariance of the formation position are achieved, which is used to guide and constrain the individual robot localization adjustment. With this method, the multiple mobile robots system shows more stable and robust on formation navigation. The simulation and physical experiment results show the feasibility, efficiency and stability of the proposed algorithm.