The hysteretic energy is an important parameter to measure the plastic cumulative damage of structures during earthquake. In this paper, with a great number of ground motions collected in the 1999 Chi-Chi, Taiwan, earthquake, the main features of constant-ductility hysteretic energy spectra is analyzed. And the effects of site condition, structural period, ductility level, damping on hysteretic energy are investigated. It is concluded that for a given period, spectra decrease with ductility ratio increases. And as the ductility ratio increase, the spectra will be closer. In the short-period region (about less than 0.8s), spectra are strongly dependent on the structural period. In the long-period region, spectra tend to be constant. The limiting period depends on the ductility ratio and the local site conditions. The damping has similar effect on spectra for all the structural period regardless ductility ratio.