Waste rubber is a typical black pollution. Producing asphalt rubber with crumb rubber is an effective way of reducing the pollution and resource utilization. The swelling and degradation degree of crumb rubber in the asphalt is a main factor of the performance of asphalt rubber. In order to study the physical and chemical behavior such as swelling and degradation, the influence of crumb rubber on the viscosity of asphalt rubber under different conditions was analyzed. And the microscopic changes of the crumb rubber were discussed by the micrographs. Test results indicate that the viscosities of asphalt rubber are increasing with the increase of the producing temperature and the dose of crumb rubber. And the viscosities are increasing firstly and then decreasing with the size decrease of the crumb rubber. The viscosity of asphalt rubber is the maximum with 60-mesh crumb rubber. The microwave can significantly improve the swelling and degradation degree of crumb rubber in the asphalt. The micrographs show that the microscopic states of crumb rubber are different in the asphalt under different conditions. The crumb rubber may only swell or swell and degrade at the same time.