In this study, the conventional Bailey-Hirsch’s relationship is extended in order to express the increase of critical resolved shear stress due to the lack of dislocation lines in a grain. This model is introduced into a triple-scale crystal plasticity model based on geometrically necessary crystal defects and the homogenization method. A FE simulation is carried out based on the proposed model for FCC polycrystals with different grain sizes. It is numerically predicted that yield behavior of fine-grained metals depends on the initial dislocation density and the initial grain size. Furthermore, yield point drop that is observed in annealed FCC fine-grained metal can be reproduced.