A novel probability hypothesis density (PHD) filter, called the Gaussian mixture convolution PHD (GMCPHD) filter was proposed. The PHD within the filter is approximated by a Gaussian sum, as in the Gaussian mixture PHD (GMPHD) filter, but the model may be non-Gaussian and nonlinear. This is implemented by a bank of convolution filters with Gaussian approximations to the predicted and posterior densities. The analysis results show the lower complexity, more amenable for parallel implementation of the GMCPHD filter than the convolution PHD (CPHD) filter and the ability to deal with complex observation model, small observation noise and non-Gaussian noise of the proposed filter over the existing Gaussian mixture particle PHD (GMPPHD) filter. The multi-target tracking simulation results verify the effectiveness of the proposed method.