An Adaptive Topic Crawler for Electronic Public Opinion
Topic crawler is a tool for collecting electronic public opinion from the internet. The identification method of topics relevance identification directly affects the acquisition rate of topic crawler. To improve the low information acquisition rate of existing topic crawlers strategy, a modified SVM classifier algorithm which is based on online incremental learning is proposed. The idea of algorithm is to remove samples that affect the training set greatly in the historical training set, and then to re-train the historical set and the incremental set to obtain a complete training set. A framework of topic crawler is constructed on the basis of this algorithm. The results of experiments show that, this method can effectively improve the acquisition rate of the crawler.
M.L. Li and G.W. Zhang
J. Fan et al., "An Adaptive Topic Crawler for Electronic Public Opinion", Advanced Materials Research, Vols. 765-767, pp. 1451-1455, 2013