In temper bead welding, toughness is one of the key criteria to evaluate the tempering effect. A neural network-based method for toughness prediction in the coarse grained heat affected zone (CGHAZ) of low-alloy steel has been investigated in the present study to evaluate the tempering effect in temper bead welding. Based on the experimentally obtained toughness database, the prediction systems of the toughness of CGHAZ have been constructed using RBF-neural network. The predicted toughness of the synthetic CGHAZ subjected to arbitrary thermal cycles was in good accordance with the experimental results. It follows that our new prediction system is effective for estimating the tempering effect in CGHAZ during temper bead welding and hence enables us to assess the effectiveness of temper bead welding.