Massive Data Analysis Based MapReduce Structure on Hadoop System

Yi Qiao Xu

doi:10.4028/www.scientific.net/AMR.981.262

Paper Titles

A Research and Exploration of Established Public Service Network Information Platform of University-Enterprise Cooperation
p.244

The Performance Evaluation of Enterprise Informatization Research Based on Dynamic Balanced Scorecard
p.249

On Normal Sequence in Abelian Group C_n⊕C_{_n}
p.255

The Application of Boosting Algorithm in Data Mining
p.258

Massive Data Analysis Based MapReduce Structure on Hadoop System
p.262

Modified Proportional 2-Tuple and its Application in Uncertainty Environment
p.267

Based on Non-Redundant Electronic Scale Engineering Development Theory
p.275

L_p-Type of Weighted Fuzzy Number Metrics Induced by Fuzzy Structured Element
p.279

An Improved Voice Activation Detection Method Based on Energy Acceleration Parameters and Support Vector Machine
p.287

HomeAdvanced Materials ResearchAdvanced Materials Research Vol. 981Massive Data Analysis Based MapReduce Structure on...

Massive Data Analysis Based MapReduce Structure on Hadoop System

Abstract:

Massive Data analysis is becoming increasingly prominent in a variety of application fields ranging from scientific studies to business researches. In this paper, we demonstrate the necessity and possibility of using MapReduce [1] module on Hadoop System [2]. Furthermore, we conducted MapReduce module to implement Clustering Algorithms [3] on our Hadoop System [4] and improved the efficiency of the Clustering Algorithms sharply. We showed how to design parallel clustering algorithms based on Hadoop System. Experiments by different size of data demonstrate that our purposed clustering algorithms have good performance on speed-up, scale-up and size-up. So, it is suitable for big data mining and analysis.