Design of the Web Log Analysis System Based on Hadoop

Yong Qing Wei; Guang Gang Zhou; Di Xu; Yu Chen

doi:10.4028/www.scientific.net/AMR.926-930.2474

Paper Titles

Bluetooth-Based Room Localization Research Based on NB and SVM Approach
p.2458

Data Storage Scheme of Sensor Network Based on HDFS
p.2462

Design and Realization of Intelligent Vehicle Based on Android Mobile Phone Bluetooth Control
p.2466

Design of the Software for Wirelessly Intercepting Voices
p.2470

Design of the Web Log Analysis System Based on Hadoop
p.2474

A Note for Order-Preserving Encryption Based on Negative Hypergeometric Distribution
p.2478

Hardware Design of Low-Power ZigBee Wireless Sensor Network Node
p.2482

HLA-Based Anti-Reconnaissance Fire-Control Radar Network Data Fusion and Control Simulation System Development
p.2486

Implement Synchronization Algorithms in OFDM System
p.2490

HomeAdvanced Materials ResearchAdvanced Materials Research Vols. 926-930Design of the Web Log Analysis System Based on...

Design of the Web Log Analysis System Based on Hadoop

Abstract:

As a result of rapid development, the Internet has become an indispensable tool in people's daily life, and web logs have been growing rapidly as well. How to deal with massive logs timely and extract information people need from the logs has become a problem; handling web log by single computer can not meet people's needs any more. Combining cloud computing and Hadoop technology, this paper established a new processing system for the collection of logs and remote parallelization analysis, which not only solved the issues of traditional systems that data handing and collection could not proceed simultaneously, but reduced the performance bottleneck of computing power and storage capacity, thereby saving much time and improving efficiency significantly.

You might also be interested in these eBooks

Progress in Applied Sciences, Engineering and Technology

View Preview

Info:

Periodical:

Advanced Materials Research (Volumes 926-930)

Pages:

2474-2477

DOI:

https://doi.org/10.4028/www.scientific.net/AMR.926-930.2474

Citation:

Cite this paper

Online since:

May 2014

Authors:

Yong Qing Wei, Guang Gang Zhou*, Di Xu, Yu Chen

Keywords:

Cloud Computing, Massive Data, Parallelization, Web Log

Export:

RIS, BibTeX

Price:

Permissions CCC:

Request Permissions

Permissions PLS:

Request Permissions

Сopyright:

Citation:

* - Corresponding Author

References

[1] T.D. HU. G, ZHOU L, KE L. Research on Hadoop-based Net work Log Analysis System . Computer Knowledge and Technology, 2010, 6(22): 6163-6164.

Google Scholar

[2] T.D. Chao Bai. Implementing of Masssive Log Analysis System Based on Parallel Computing . Computer Technology and Development, Vol. 23 (2013) No. 7, p.80. (In Chinese).

Google Scholar

[3] Cloud Computing Security: making Virtual Machines Cloud-Ready [R]. www. cloudready-security. com, (2008).

Google Scholar

[4] Shvachko K, Kuang H, Radia S, et al. The Hadoop distributed file system[C]/Mass Storage Systems and Technologies (MSST), 2010 IEEE 26th Symposium on. IEEE, 2010: 1-10.

DOI: 10.1109/msst.2010.5496972

Google Scholar

[5] Kai Gao, A net work packet analysis and design of software development . Net land, 2013, (In Chinese).

Google Scholar

[6] T.D. Z. Y Zhang. Research of Grid-Based TCP Log Two-Step Clustering Algorithm . Journal of university of jinan, Vol. 25 (2011) No. 2, p.196. (In Chinese).

Google Scholar

[7] H. Y Zhang. The Cloud Computing Based on Hadoop Platform and Log Analysis, (Harbin University of Science and Technology China 2012) (In Chinese).

Google Scholar