In this paper, a new feature fusion method for Handwritten Character Recognition based on single tri-axis accelerometer has been proposed. The process can be explained as follows: firstly, the short-time energy (STE) features are extracted from accelerometer data. Secondly, the Frequency-domain feature namely Fast Fourier transform Coefficient (FFT) are also extracted. Finally, these two categories features are fused together and the principal component analysis (PCA) is employed to reduce the dimension of the fusion feature. Recognition of the gestures is performed with Multi-class Support Vector Machine. The average recognition results of ten Arabic numerals using the proposed fusion feature are 84.6%, which are better than only using STE or FFT feature. The performance of experimental results show that gesture-based interaction can be used as a novel human computer interaction for consumer electronics and mobile device.