Research and Design of a Massive Offline Data Analysis System Based on Hadoop

Sen Pan; Li Peng Zhu; Bin Hu; Pei Yang

doi:10.4028/www.scientific.net/AMM.631-632.1049

Paper Titles

Research on the Key Technologies of Highly Reliable Embedded IDE Based on R80515
p.1027

Structured Big Data Management System Supported Cross-Domain Query
p.1033

The Key Technology Research of Mechanical Parts Dynamic Management Database System
p.1039

AG-Index: Adjacent Edge Hash Index for Graph Databases
p.1045

Research and Design of a Massive Offline Data Analysis System Based on Hadoop
p.1049

Research on Data Mining Optimization and Security Based on MapReduce
p.1053

A Novel over Writable and Restoring Solution of Filesystem for NAND Flash
p.1057

Data Provenance Tracing for Transformation Diagram Based on Wivern
p.1061

Analysis of the Construction of the Library Information Resources from the Perspective of Big Data
p.1067

HomeApplied Mechanics and MaterialsApplied Mechanics and Materials Vols. 631-632Research and Design of a Massive Offline Data...

Research and Design of a Massive Offline Data Analysis System Based on Hadoop

Abstract:

According to the rapid growth of the scale of the data at present, the traditional model of data analysis based on stand-alone can't meet to the storage and processing of massive data. With the rise of big data technology, integrating the traditional method of data analysis with big data platform to improve the efficiency of data analysis has become a research direction. This paper analyzes the integration of offline data processing and the Hadoop HDFS, MapReduce, designs a large offline data analysis system platform based on Hadoop, and introduces the core function module.

You might also be interested in these eBooks

View Preview

Info:

Periodical:

Applied Mechanics and Materials (Volumes 631-632)

Pages:

1049-1052

DOI:

https://doi.org/10.4028/www.scientific.net/AMM.631-632.1049

Citation:

Cite this paper

Online since:

September 2014

Authors:

Sen Pan*, Li Peng Zhu, Bin Hu, Pei Yang

Keywords:

Analysis, Hadoop, HDFS, MapReduce, Massive Data

Export:

RIS, BibTeX

Price:

Permissions CCC:

Request Permissions

Permissions PLS:

Request Permissions

Сopyright:

Citation:

* - Corresponding Author

References

[1] Jie Cui, Taoshen Li, Hongxing Lan: Design and Development of the Mass Data Storage Platform Based on Hadoop[j]. Journal of Computer Research and Development, 2012, (z1): 12-18.

Google Scholar

[2] Ying Chen, Yunyong Zhang, Lei Xu: Research on Large-Scale Data Processing Based on Hadoop and Relational Database[J]. Telecommunications Science, 2010, (11): 47-50.

Google Scholar

[3] Chensheng Fan: Design and Implementation of Mass Data Analysis System Based on Hadoop[D]. Xidian University, (2012).

Google Scholar

[4] Hongxia Xia: The Research on the Technology of Computing and Storage the Mass Data Based on Hadoop Cluster[D]. Wuhan University of Technology, (2012).

Google Scholar

[5] Sen Pan, Jiangfeng She, Bo Liu: Constructing Urban Archives Management System Based on WebGIS[J]. Zhejiang Archives, 2009, 11, 26-28.

Google Scholar

[6] Information on http: / hadoop. apache. org.

Google Scholar

[7] Pavlo A, Paulson E, Basin A: A comparison of approaches to large-scale data analysis[A]. New York, USA, (2009).

Google Scholar