A novel median filter based manifold ranking approach for robust image annotation was presented in this paper. Firstly, we propose a similarity matching approach of manifold ranking to improve the robustness of traditional label propagation with median filter. Then the refined affinity matrix was applied to label propagation. Thus, the traditional pair-wise similarity is updated with scores using median filter of its neighbors in manifold space. Finally, the proposed image annotation approach propagated the image labels from the labeled images to the whole image database. The experiments over Corel images have shown that embedding median filter approach in manifold space is beneficial in image annotation.