The objective of this study is to develop a numerical model based on microstructural images of concrete and fundamental material properties of each constituent of concrete subjected to alkali-silica reaction (ASR). A microstructure-based finite element approach is employed directly to analyze the mechanical response of concrete to ASR. The modeling work involves acquiring and processing of microstructural images of specimens suffering from ASR using scanning electron microscopy, and implementing finite element program to analyze the microstructural images. The formulation of this model is based on pressure caused by the ASR product and on properties such as Young’s modulus and Poisson’s ratio. The finite element analysis program used to simulate structural behavior of structures attacked by ASR is object-oriented finite element developed at National Institute of Standards and Technology. The numerical results from this model are compared with experimental data, which have been measured using ASTM standard test C1260. The results show that the development and widening of cracks by formation and swelling of ASR gel cause the majority of expansion of mortar specimens rather than elastic elongation due to gel swelling.