The Constructing of Test Set on Chinese Information Retrieval

Ming Yao Zhang; Dong Hong Ji

doi:10.4028/www.scientific.net/AMR.546-547.645

Paper Titles

Discussion of Classification for Imbalanced Data Sets
p.622

Correlation Analysis Method and its Application to Interpretation of Regional Gravity and Magnetic Anomalies in Eastern Xinjiang, China
p.628

The Analysis and Review of Mobile Surveillance Video Based on AVS-S
p.634

Elementary Discussion on Data Management of the Internet of Things
p.640

The Constructing of Test Set on Chinese Information Retrieval
p.645

BPMO-Based OWL Representation of Business Processes
p.651

Application to Sedimentary Facies Identification Used RGB Fusion Imaging in Multi-Attribute
p.656

Application of the Data Fusion Method Based on Fuzzy Theory and Evidence Theory in Vehicle Recognition
p.661

The Comparison of Different Chromosome Coding Method Based on Genetic Algorithm
p.666

HomeAdvanced Materials ResearchAdvanced Materials Research Vols. 546-547The Constructing of Test Set on Chinese...

The Constructing of Test Set on Chinese Information Retrieval

Abstract:

In this paper, we propose a Template-Matching strategy combined with statistics in the tasks of human evaluation of NTCIR to avoid the arbitrariness of human evaluators. By practical operation and within-system checkup, the strategy helps avoid the arbitrariness of human evaluators and provides a benchmark for human evaluation of NTCIR.

You might also be interested in these eBooks

Electrical Insulating Materials and Electrical Engineering

View Preview

Info:

Periodical:

Advanced Materials Research (Volumes 546-547)

Pages:

645-650

DOI:

https://doi.org/10.4028/www.scientific.net/AMR.546-547.645

Citation:

Cite this paper

Online since:

July 2012

Authors:

Ming Yao Zhang, Dong Hong Ji

Keywords:

Arbitrariness, Benchmark, Template-Matching Strategy, Test Data Set

Export:

RIS, BibTeX

Price:

Permissions CCC:

Request Permissions

Permissions PLS:

Request Permissions

Сopyright:

Citation:

References

[1] Kando, N., Kuriyama, K., Nozue, T., et al.: Overview of IR Tasks at the First NTCIR Workshop. Proceedings of NTCIR-1 Workshop Meeting, August 30 - September 1, 1999, Tokyo, Japan. 1999: pp.11-34.

DOI: 10.1145/312624.312730

Google Scholar

[2] Kando, N.: Overview of the Seventh NTCIR Workshop. Proceedings of NTCIR-7 Workshop Meeting, December 16–19, 2008, Tokyo, Japan. 2008: pp.1-9.

Google Scholar

[3] Mitamura, T., Nyberg, E., Shima, H., et al.: Overview of the NTCIR-7 ACLIA Tasks: Advanced Cross-Lingual Information Access. Proceedings of NTCIR-7 Workshop Meeting, December 16–19, 2008, Tokyo, Japan. 2008: pp.11-25.

Google Scholar

[4] Sakai, T., Kando, N., Mitamura, T. et al.: Overview of the NTCIR-7 ACLIA IR4QA Subtask. Proceedings of NTCIR-7 Workshop Meeting, December 16–19, 2008, Tokyo, Japan. 2008: pp.1-11.

Google Scholar

[5] Ferro, N., Peters, C.: From CLEF to TrebelCLEF: the Evolution of the Cross-Language Evaluation Forum. Proceedings of NTCIR-7 Workshop Meeting, December 16–19, 2008, Tokyo, Japan. 2008: pp.557-593.

Google Scholar

[6] Harman, D. K.: Overview of the First Text REtrieval Conference (TREC1). The First Text REtrieval Conference (TREC1). Gaithersburg, MD: U.S. Dept. of Commerce, National Institute of Standards and Technology. 1993: pp.1-20.

DOI: 10.6028/nist.sp.500-207

Google Scholar

[7] Deng, Y., Xu, B.J., Liu, S., et al.: Answer Path at NICIR-7 CCLQA Track. Proceedings of NTCIR-7 Workshop Meeting, December 16-19, 2008, Tokyo, Japan. 2008: pp.60-65.

Google Scholar

[8] http: /aclia. lti. cs. cmu. edu: 8080/epan/he/ir_evaluation. jsp.

Google Scholar

[9] http: /wenda. tianya. cn/wenda/thread?tid=0d699255ebc0c8ef.

Google Scholar

[10] Zhang, X. K., Zhao, D.Q.: An Overview on TREC and Its Latest Development. Intelligence Theory and Application. February 16, 2004. Beijing, China. 2004: pp.537-540.

Google Scholar

[11] Zheng, D.Q., Li, S., Zhao, T.J.: CLIR Model Based on a Combination of Ontology and Statistical Method. Journal of Harbin Institute of Technology. January 2008. Harbin, China. 2008: pp.77-80.

Google Scholar

[12] Tang, Y., Lin, L.X., Luo, Y. M. et al.: Survey on Information Retrieval System based on Question Answering System. Computer Applications. November, 2008. Beijing, China. 2008: pp.2745-2748.

DOI: 10.3724/sp.j.1087.2008.02745

Google Scholar

[13] http: /en. www. wikipedia. org.

Google Scholar