The medium and heavy plate of 800 MPa grade ultra-fine grained steel was welded by CO2 gas shielded welding using large line energy. The effect of cooling condition on microstructure and mechanical properties of welded joints was investigated. The results showed that the cases about significant grain size increasing and strength decreasing do exist in the heat affected zone of large line energy welded joints of 800 MPa grade ultra-fine grained steel. Grain growth and softening in the heat affected zone could be suppressed effectively by water cooling in the course of welding. The mechanical properties of welded joints could be significantly increased by water cooling, and the process of CO2 gas shielded welding under water cooling is practical for joining ultra-fine grained steel using large line energy.