Interactions between adjacent, insufficiently separated buildings have been repeatedly observed during major earthquakes. This phenomenon, known as the earthquake-induced structural pounding, may be the reason of local damage at the contact points as well as may lead to the extensive damage at the base of the colliding structure or even initiate its total collapse. In this paper, we examine the importance of inelastic modelling of structural behaviour as the result of damage due to earthquake excitation and structural pounding. The study concerns two adjacent four-storey buildings with different dynamic properties. In the numerical simulations, the nonlinear viscoelastic model is used to model the pounding force during collisions at different storey levels of the structures. The model allows us to take into consideration the dissipation of energy due to damage taking place at the time of collision. Three different ground motion records with different peak ground acceleration levels are used in the study. The comparison between elastic and inelastic damage-involved structural behaviour is investigated. The results of the study show significant changes in the dynamic responses of the inelastic systems as compared to those of elastic ones. The results clearly indicate that modelling the colliding buildings to behave inelastically is really essential in order to obtain accurate damage-involved structural response under earthquake excitation.