Coal gangue is the waste of coal mining. It is discharged and stacked all around the mine and the dust pollution has become a public hazard. This paper commits an investigation on the asphalt mixture which takes advantage of coal gangue as coarse aggregate. The physical-mechanical performances of coal gangue aggregate are presented and the result indicates that coal gangue aggregate can meet the specifications except the content of the Flat and Elongate particles (F&E content). By controlling the rational content of F&E in the mixture, different types of asphalt mixes with different percentage of coal gangue coarse aggregate are tested and the overall performance of asphalt mixture is investigated. The result testifies that the asphalt mixture containing coal gangue has the equivalent performance with limestone mixture. It is observed that the coal gangue used as coarse aggregate can guarantee the properties of HMA. Thus the HMA can be used in asphalt pavement.