The Review of Automatic Chinese Word Segmentation

Abstract:

As the fundamental work of Chinese information processing, Chinese word segmentation has achieved great progress since its birth. This paper reviews the research status of the CWS, discusses the formal model of automatic word segmentation, and analyzes the difficulties of word segmentation.

• Periodical: Applied Mechanics and Materials (Volumes 701-702)
• Pages: 386-389
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.701-702.386
• Online since: December 2014
• Authors: Xiao Yan Ren, Yun Xia Fu
• Keywords: Chinese Information Processing, Chinese Word Segment (CWS), Segment Methods, Syntax Analysis

References

[1] Zhao H, Zhu H. Research of Chinese Word Segmentation Based on Double-Array Trie; Journal of Hunan University(Natural Sciences), 2009 36(5): pp.77-80.

[2] Zhao Z, Chen T, Zhu L. A new word segmentation strategy based on phraseology; Journal of Suzhou University (Natural Sceince) , 2002 18(3): pp.44-48.

[3] Mo J, Zheng Y, Shou z etc, Improved Chinese word segmentation method based on dictionary. Computer Engineering and Design, 2013 34(5): pp.1802-1807.

[4] Xue Nianwen. Labeling Chinese Predicates with Semantic Roles. Computational Linguistics . (2008).

[5] Li Jiafu, Segmenting Chinese Based on Probabilistic Model, Journal of System Simulation, 2002 14(5): pp.544-547.

[6] Liu Shanfeng etc, Chinese Character Based Segmentation Using Hidden Markov Model, the 12th National Conference on Man-Machine Speech Communication.

[7] ZHAO Hai, Chunyu Kit, Effective Subsequence-Based Tagging for Chinese Word Segmentation. Journal of Chinese Information Processing, 2007 21(5): 8-13.

[8] Huang Degen, Jiao Shidou, and Zhou Huiwei, Dual-Layer CRFs Based on Subword for Chinese Word Segmentation. Journal of computer research and development 2010 47(5): 962-968.

[9] Zhou Jun-sheng; Dai Xin-yu; Ni Rui-yu; Chen Jia-jun, A Hybrid Approach to Chinese Word Segmentation around CRFs, Proceedings of the Fourth SIGHAN Workshop on Chinese Language Processing, Jeju Island, Korea, October 14, (2005).

[10] Dong Song and Anoop Sarkar. Voting between Dictionaray-based and Subword Tagging Models for Chinese Word Segmentation [ C]/Proceedings of the Fifth SIGHAN Workshop on Chinese Language Processing. Sydney, Australia, 2006: 126-129.

[11] Ruiqiang Zhang, Genichiro Kikui and Eiichiro Sum-ita. Subword-based tagging for confidence-dependent Chinese word segmentation[C]/Proc. of the COLING/ACL on Main conference poster sessions. Sydney, Australia, 2006: 961-968.

DOI: 10.3115/1273073.1273196

[12] LUO Yanyan, HUANG Degen, Chinese Word Segmentation Based on the Marginal Probabilities Generated by CRFs , JOURNAL OF CHINESE INFORMATION PROCESSING, 2009 v23(5): 3-8.

[13] HUANG Chang-ning, ZHAO Hai, Chinese Word Segmentation: A Decade Review; JOURNAL OF CHINESE INFORMATION PROCESSING; 2007 v21(3): 8-19.

[14] andi wu . Chinese word segmentation in msr-nlp[a]. in: proceedings of the second sighan workshop on Chinese language processing [c]. Sapporo, japan : july 11-12, 2003, 172-175.

DOI: 10.3115/1119250.1119277

[15] YANG Xiao-jia, JIANG Wei, HAO Wen-ning, Implementation of Field Word SegmentationBased on Ontology and Syntax Analysis, computer engineering, 2008, v34(23): 26-28.

[16] LAI Shiwei, XU Lisheng, CHEN Yubo, LIU Kang etc, Chinese Word Segment Based on Character Representation Learning. Journal of Chinese Information Processing, 2013 v27(5) : 8-14.

[17] Zheng Xiaogang, Han lixin etc; a Combination-Type Chinese Word Segmentation Method ; Computer Applications and Software 2012 29(7) : 26-29.