The Research of Knowledge-Based Chinese Segmentation Method

Abstract:

Current Chinese segmenting method doesn’t consider grammar and semantics during the process of segmenting. So segmenting accuracy and speed are not good. Therefore, on the base of analyzing the peculiarity of Chinese, a knowledge-based Chinese segment method, KSM, is presented. It uses a dictionary of hierarchical structure to get all the re-segmented words and makes ambiguity processing easy and effective. It gets support not only from letters and words knowledge, but from grammar and semantics knowledge. It removes wrong segmentation by checking grammar and semantics. Also It can learn new words to improve the accuracy of segmenting.

• Periodical: Applied Mechanics and Materials (Volumes 651-653)
• Pages: 2545-2548
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.651-653.2545
• Online since: September 2014
• Authors: Guang Ming Zhou*
• Keywords: Ambiguity-Processing, Artificial Intelligence, Chinese Segmenting, Natural Language Process, Syntax Semantics

References

[1] Liu yin, Error Driven Learning to Chinese Segmentation Based on Rules, The journalism of Qinghua University, vol. 27, pp.20-25, January (1999).

[2] Hou Guofeng, The Implementation and Design of a Natural Language Understanding System, Computer Application and Research, pp.35-38, Feb. (2002).

[3] Yang Shu, Chinese Segmentation Error-correcting Method, Computer Science, pp.17-21, May1989.

[4] Lu Xiangyuan, The Method of Modern Chinese Grammar Learning, Beijing Academic Press, (1985).

[5] Allen James, Natural Language Understanding, The Benjamin/Cummings Publishing Company, (1987).

[6] Wu Quanyuan, Artificial Intelligence and Expert System, Changsha: University of Defense Technology Press, 1995, p.135–180.