In this paper, a time domain structural damage detection approach based on Kalman filter and least square estimation is proposed for structures under limited input and output measurements. First of all, the dynamic parameters of the structure, such as stiffness and damping coefficients are identified by sequentially utilizing the extended Kalman estimator and the least squares estimation. Secondly, after the structural dynamic parameters are gained, the normal Kalman filter and least square estimation is used to identify the state vector and unknown input of the structure, and the changes of the structure such as reducing of the stiffness are regarded as ‘additional unknown input’. Then the changed parameters can be calculated by analyses the connection between ‘additional unknown input’ and changing parameters, Local structural damage in the structure can also be detected by tracking the changes in the identified values of structural dynamic parameters in time domain at element level, e.g., the degrading of stiffness parameters. Numerical example of detecting structural local damages in a four story share building illustrates the efficiency of the proposed approach.