The paper deals with analysis of contact stress fields and relative displacements as well as inspection of fretting phenomena in the neighborhood of mating surfaces of the riveted joint subjected to cyclic loading. The study of micro-local phenomena follows the riveting process analysis. Numerical FE simulations of the upsetting process are carried out to determine the residual stress and strain fields. Nondestructive testing methods are used for validation of numerical results. The contact with friction is defined between the mating parts of the joint. The influence of the initial load and sheet material model is studied. Maximum values of relative displacements are comparatively small but surface condition is strongly affected by sliding movements during cyclic loading. Plastic strain energy is taken into account for more efficient numerical analysis of the fretting wear.