A new mode of fault monitoring and controling methods on rotating speed fluctuation was proposed in this study. Torsional vibration model and identification equation of speed fluctuation of large-scale rotary machinery was established based on a two-mass motor driving model. Tachogenerator was adopted to measure speed fluctuation in torsional vibration experiment of large-scale rotary machinery. According to the short time fourier transforms method, the non-steady cyclical or quasi-cyclical characteristics signal of rotating speed fluctuation on elastic shafts were transformed into steady signal to study in a fixed time window function. The methods of monitoring rotating speed fluctuation developed nonlinear stable state signal processing into linear short time fourier transforms signal. The real rotating speed fluctuation solution could be obtained after the data of signal acquisition post-processing by the methods of frequency spectrum analysis and modal analysis. Based on data of signal acquisition, using the methods of fourier phase frequency spectrum, logarithm amplitude frequency spectrum, and self-power spectrum, the quantitative expression under the quantitative analysis stable state was obtained. Through the introduction of realtime signal on rotating speed fluctuation to feedback control system, it is easy to program to realize the real-time on-line torsional vibration monitor of the complex mechanism transmission system on the large scale rotating machinery.