Research of Distributed Search Engine Based on Hadoop

Zhen Quan Wu; Bing Pan

doi:10.4028/www.scientific.net/AMM.631-632.171

Paper Titles

Review of Regional Power Grid Fault Information Screening
p.147

Single-Step Optimal Design for a Class of Nonlinear Singular System and Epidemic Control
p.153

Trust Calculation Model in Knowledge Community
p.161

Enhancing Endpoint Trusted Connect Using Lightweight Virtualization
p.165

Research of Distributed Search Engine Based on Hadoop
p.171

One Fault Diagnosis Method Based on Fuzzy Equal Relationship and Rough Set Theory
p.175

Research on Application of Google Glass in the Field of Journalism
p.180

The Research of Smart Home Robots Based on 3G Mobile Network
p.184

Design and Implementation of Intelligent Greenhouses Based on the Internet of Things
p.188

HomeApplied Mechanics and MaterialsApplied Mechanics and Materials Vols. 631-632Research of Distributed Search Engine Based on...

Research of Distributed Search Engine Based on Hadoop

Abstract:

Combined with the Map/Reduce programming model, the Hadoop distributed file system, Lucene inverted file indexing technology and ICTCLAS Chinese word segmentation technology, we designed and implemented a distributed search engine system based on Hadoop. By testing of the system in the four-node Hadoop cluster environment, experimental results show that Hadoop platform can be used in search engines to improve system performance, reliability and scalability.

You might also be interested in these eBooks

View Preview

Info:

Periodical:

Applied Mechanics and Materials (Volumes 631-632)

Pages:

171-174

DOI:

https://doi.org/10.4028/www.scientific.net/AMM.631-632.171

Citation:

Cite this paper

Online since:

September 2014

Authors:

Zhen Quan Wu, Bing Pan

Keywords:

Distributed Search Engine, Hadoop, HDFS, MapReduce

Export:

RIS, BibTeX

Price:

Permissions CCC:

Request Permissions

Permissions PLS:

Request Permissions

Сopyright:

Citation:

References

[1] Chenxi Fan: The Research and Application of Search Engine based on Hadoop (In Chinese) [D]. Electronics and communication engineering field, (2013).

Google Scholar

[2] Junsheng Wang, Yunmei Shi, Yangsen Zhang: Journal of Beijing Information Science and Technology University, Vol. 26 No. 4 (2011), pp.53-56.

Google Scholar

[3] Jeff Dean. Experiences with MapReduce, an Abstraction for Large-Scale Computation [R]. Proc. 15th. International Conference on Parallel Architectures and Compilation Techniques, (2006).

DOI: 10.1145/1152154.1152155

Google Scholar

[4] Owen O'Malley. Programming with Hadoop's Map/Reduce [R]. ApacheCon EU, (2008).

Google Scholar

[5] Yuzhong Cao. Distributed parallel programming with Hadoop(In Chinese), Part 1 [EB / OL]. http: /www. ibm. com/developerworks/cn/opensource/os-cn-hadoop1/, 2014-05-07.

Google Scholar

[6] ICTCLAS Chinese word segmentation system on http: /ictclas. org/, 2014-05-06.

Google Scholar