This paper proposes an improved particle swarm optimization algorithm (PSO) for the global and local equilibrium problem of searching ability. It improves the iterative way of inertia weight in PSO, using non-linear decreasing algorithm to balance, then PSO combines with simulated annealing (SA). Finally, the optimization test experiments are carried out for the typical functions with the algorithm (ULWPSO-SA), and compare with the basic PSO algorithm. Simulation experiments show that local search ability of algorithm, convergence speed, stability and accuracy have been significantly improved. In addition, the novel algorithm is used in the parameter optimization of support vector machines (ULWPSOSA-SVM), and the experimental results indicate that it gets a better classification performance compared with SVM and PSO-SVM.