Carbon steel is the most popular engineering material, usually consisted of ferrite and cementite phases. Internal stress state of the steel under thermal or mechanical loading is strongly affected by the amount and morphology in the cementite phase. With this aim, a computational model which applies the finite element method at the microscale was used in present study. Effects of volume fraction and particle size of the spheriodal cementite on the internal stress states in carbon steels under the mechanical and thermal loadings are investigated. To verify the reliability of the computational simulations, the residual stresses in the constituent phases are measured by means of X-ray stress diffraction technique. The computational simulations fit well with the experimental data, and the microstructure-based model is validated.