According to the limitations of calculation of the original random early detection (RED) algorithm in linear packet loss rate. This paper proposes an improved algorithm which imposes nonlinear smooth for packet loss rate function of RED algorithm. The speed of growth of packet loss rate is relatively slow near the minimum threshold, while near the maximum threshold the speed of growth of packet loss rate is relatively faster. In this case, using the trend of the average queue length to dynamically adjust the parameters of the RED algorithm, it reduces the dependence on the parameters of the RED algorithm and enhances the stability of the algorithm. NS simulation shows that this algorithm has been significantly improved for packet loss rate, throughput and other performance.