Hyperspectral data is endowed with characteristics of intrinsic nonlinear structure and high dimension. In this paper, a nonlinear manifold learning algorithm - ISOMAP is applied to anomaly detection. Then an improved ISOMAP algorithm is developed based on the analysis of the inherent characteristics of hyperspectral imagery. The improved ISOMAP algorithm selects neighborhood according to a novel measure of combination of spectral gradient and spectral angle in order to make the algorithm more robust to the changes of light and terrain. Experimental results prove the effectiveness of the algorithm in improving the detection performance.