Spatial verification for object retrieval is often time-consuming and susceptible to viewpoint changes. In this paper, we propose a novel spatial verification method that is robust to viewpoint changes. Firstly, the affine covariant neighborhoods (ACNs) of corresponding local regions are matched to eliminate possible false matches. Secondly, the RANSAC is performed to estimate the affine transformation from each single pair of corresponding local regions without the gravity vector assumption used in previous spatial verification methods. Experimental results demonstrate that this method is more robust and fast than previous spatial verification methods.