Design and Implementation of Massive Data Retrieving Based on Cloud Computing Platform

Wei Xiao; Chun Lei Ji; Jian Dun Li

doi:10.4028/www.scientific.net/AMM.303-306.2235

Paper Titles

A GIS Based Assistant Information System for Forest Fire Prevention Direction
p.2215

A Method of Geospatial Data Files Conversion Based on Semantic
p.2221

A Novel Discrete Differential Evolution Algorithm for SVRPSPD
p.2227

Cloud-Based Bayesian Inference for Online People Health Status Assessment
p.2231

Design and Implementation of Massive Data Retrieving Based on Cloud Computing Platform
p.2235

Dynamic Load Balancing Algorithm Based on Task Classification for Data Retrieval
p.2241

Fast Tracking for Video Target Tracking
p.2245

General Design and Implementation of Heterogeneous Data Conversion Tools
p.2249

Research of Object-Oriented 3D Video Coding Segmentation
p.2254

HomeApplied Mechanics and MaterialsApplied Mechanics and Materials Vols. 303-306Design and Implementation of Massive Data...

Design and Implementation of Massive Data Retrieving Based on Cloud Computing Platform

Abstract:

Considering the low efficiency of massive data retrieving in traditional parallel processing, by taking advantage of the great availability of cloud computing paradigm, we propose a hybrid solution based on Map-Reduce model and distributed computing framework--Spark. Moreover, we design and implement this solution in our lab. The results show that the solution can effectively improve the performance of massive data retrieving.

You might also be interested in these eBooks

Sensors, Measurement and Intelligent Materials

View Preview

Info:

Periodical:

Applied Mechanics and Materials (Volumes 303-306)

Pages:

2235-2240

DOI:

https://doi.org/10.4028/www.scientific.net/AMM.303-306.2235

Citation:

Cite this paper

Online since:

February 2013

Authors:

Wei Xiao, Chun Lei Ji, Jian Dun Li

Keywords:

Cloud Computing, MapReduce, Massive Data Retrieving, Spark

Export:

RIS, BibTeX

Price:

Permissions CCC:

Request Permissions

Permissions PLS:

Request Permissions

Сopyright:

Citation:

References

[1] 2010 Digital Universe Study [EB/OL]. http: /gigaom. files. wordpress. com/2010/05/2010-digital-universe-iview-5-4-10. pdf. 2010. 7. 27.

Google Scholar

[2] SCHEN Kang, Weimin ZHENG. Cloud Computing: System Instances and Current Research [J]. Journal of Software, vol. 20, no. 5, 2009, pp.1337-1348. (In Chinese).

Google Scholar

[3] Peng WANG, Huafeng HUANG, Qin CAO. Cloud Computing: China Future IT Tactic [M]. Beijing: People's Post and Telecom Press, 2010. (In Chinese).

Google Scholar

[4] Armbrust M, Fox A, Grifth R, et al. Above the Clouds: A Berkeley View of Cloud Computing[R] / Technical Report No. UCB/ EECS-2009-28. Berkeldy: Department of Electrical Engineering and Computer Sciences, University of California, (2009).

Google Scholar

[5] Lizhe WANG, Gregor von LASZEWSKI, Andrew YOUNGE, Xi HE. Cloud Computing: a Perspective Study. Nanyang Technological University. 2009, no. 14.

Google Scholar

[6] M. Armbrust, A. Fox, R. Griffith et al. A view of cloud computing [J]. Communications of the ACM, vol. 53, no. 4, pp.50-58, (2010).

Google Scholar

[7] R. Buyya, C. S. Yeo, S. Venugopal et al. Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility [J]. Future Generation Computer Systems, vol. 25, no. 6, pp.599-616, (2009).

DOI: 10.1016/j.future.2008.12.001

Google Scholar

[8] Spark: http: /www. spark-project. org.

Google Scholar

[9] http: / bbs. Sciencenet. cn/cn/hom. php? mod=space&uid=425672&do=blog&id= 520947.

Google Scholar

[10] Apache Mesos. http: / incubator. Apache. Org mesos.

Google Scholar

[11] J Dean, S Ghemawat. Communications of the ACM, vol. 51, no. 1, 2008, pp.107-113.

Google Scholar

[12] Jeffrey Deanand Sanjay Ghemawat. MapReduce: Simplified DataProcessing on Large Clusters. (2004).

Google Scholar

[13] Gluster: http: /www. gluster. org.

Google Scholar