Performing an event-based continuous kinetic Monte Carlo (KMC) simulation, all the important kinetic behaviors take place during the growth of the semiconductor material in the molecular beam epitaxy (MBE) system such as deposition, diffusion, desorption, and nucleation are considered, we investigate the effects of the growth conditions which are important to form semiconductor quantum dot (QD) in MBE system. The simulation results provide a detailed characterization of the atomic kinetic effects. The KMC simulation is also used to explore the effects of anisotropy effects to the epitaxy growth of QD. We find that the flux plays an important role in determining the size of the QD. The agreement between our simulation and experiment indicates that this KMC simulation is useful to study the growth mode and the atomic kinetics during the growth of the semiconductor QDs in MBE system.