For better understanding on adhesive ability and action mechanism of asphalt mortar, a series of mortar based on Zhong-hai asphalt were prepared and evaluated by the method used in high molecule composite materials. Some influencing factors including temperature, filler-binder ratio and asphalt type were discussed on interface adhesive ability. And some conclusions were conducted. The interface adhesive ability between asphalt and mineral filler will decline as temperature fall, until the glass transition or interface phase break appears in mortar system. To the same mineral filler and asphalt, interface adhesive ability of mortar will declines as filler-binder-ratio increases under a certain temperature, and beyond this temperature, high filler-binder-ratio will upgrade the interface adhesive ability and the whole binding performance of mortar.