As non-contact seal, labyrinth seal is widely used in rotor system of high speed. However, with the development of turbo-machinery toward high performance and huge capacity, gas excitation vibration within the labyrinth seal becomes the main reason threatening safe operating of machinery sets. Recently three dimensional computational fluid dynamic was applied to analyse inside flow in labyrinth seal. These researches, while greatly improve rotordynamic prediction of labyrinth seals, are mostly focused in steady calculation. In fact, rotor inside the seal is whirling in a nonlinear behaviour, which makes the flow unstable. In an effort to analyse the non-linear behaviour of flow in labyrinth seal, this paper utilizes an improved dynamic mesh technology to realize unsteady calculation. The Reynolds Averaged Navier Stokes equations is solved by a commercial CFD program, FLUENT. Steady calculations are firstly done to determine mesh density and turbulence model, then an unsteady analysis is used to study gas excitation force. The influence of initial condition to the unsteady analysis is discussed. This method allows modeling of rotor orbit around the eccentric position and gives prediction of nolinear gas excitation force.