Although ultimate boundedness of several classes of neural networks with constant delays was studied by some researchers, the inherent randomness associated with signal transmission was not taken account into these networks. At present, few authors study ultimate boundedness of stochastic neural networks and no related papers are reported. In this paper, by using Lyapunov functional and linear matrix inequality, some sufficient conditions ensuring the ultimate boundedness of stochastic neural networks with time-varying delays are established. Our criteria are easily tested by Matlab LMI Toolbox. One example is given to demonstrate our criteria.