Heat treatment is an effective means of structural adjustment and performance improvement of AlN ceramics. AlN ceramics prepared at high pressure with Y2O3 as a sintering aids were heat treated in a nitrogen flow atmosphere. The effects of heat treatment on microstructure and thermal conductivity of AlN ceramics were studied. The results show that the grain size of the AlN ceramics heat-treated at 970 °C for 2 h is significantly increased, the actual crystal morphology is realistic and the second phases are almost present at the grain boundaries or triple pockets compared with the samples without heat treatment, which thermal conductivity has reached 156.7 W/(m·K) from 77.3 W/(m·K). However, the pore size of AlN ceramics is increased and there is the phenomenon of anti-densification while the heat treatment time is extended to 4 h. The thermal conductivity of AlN ceramics heat-treated at 970 °C for 4 h is reduced to 92.6 W/(m·K).