Gabor transform is very suitable for time-frequency analysis and good for filtering non-stationary signals. The threshold of the Gabor transform filter is a key factor for the filter’s effectiveness. A novel threshold based on initial highest inter-cluster distance probability (IH-ICDP) is described in this paper and it can make the filter more efficient. Some experiments have been carried out under several conditions to evaluate the new threshold’s characteristics. The experimental results show that Gabor transform filter with this proposed threshold works better than wavelet transform filter, especially when the signal’s SNR is very low. From the evaluation results, it is possible to consider that the threshold presented is optimal or nearly optimal.