The Research and Improvement of MapReduce Cluster Scheduling Strategy Based on Ant Colony Optimization

Ya Ping Zhang; Jun Qin; Zhao Zhai

doi:10.4028/www.scientific.net/AMM.513-517.1989

Paper Titles

Hierarchical Certificate-Based Encryption: Definition and an Efficient Construction
p.1971

The S3C2440’s Camera Interface Driver Development Based on Embedded Linux
p.1975

Research and Simulation of Performance of W-SVD Watermarking Algorithm Based on Color Images
p.1980

The Architecture Design of Information Dispersed Storage System Based on the RAIC
p.1984

The Research and Improvement of MapReduce Cluster Scheduling Strategy Based on Ant Colony Optimization
p.1989

Automatic Summarization Based on Mutual Information
p.1994

A Method to Creating Visualization Model with Consistency Rule in Paradigm
p.1998

Web GIS Research and Application Based on SVG
p.2004

The Realization of HARQ Based on UE in the Uplink of LTE System
p.2008

HomeApplied Mechanics and MaterialsApplied Mechanics and Materials Vols. 513-517The Research and Improvement of MapReduce Cluster...

The Research and Improvement of MapReduce Cluster Scheduling Strategy Based on Ant Colony Optimization

Abstract:

The distributed computing framework of MapReduce has been widely used in big companies as a powerful tool for processing large scale of data. This paper will introduce the existing algorithms about MapReduce job scheduling and analyze the major two ones. It points out the defects of multi-task scheduling application on processing massive jobs and proposes a kind of multi-task cluster schedule algorithm MSBACO which is based on the ant colony optimization. The results of the experiments have proved its effectiveness and stability in heterogeneous environment.

You might also be interested in these eBooks

View Preview

Info:

Periodical:

Applied Mechanics and Materials (Volumes 513-517)

Pages:

1989-1993

DOI:

https://doi.org/10.4028/www.scientific.net/AMM.513-517.1989

Citation:

Cite this paper

Online since:

February 2014

Authors:

Ya Ping Zhang*, Jun Qin, Zhao Zhai

Keywords:

Ant Colony Optimization, MapReduce Clusters, Schedule Strategy

Export:

RIS, BibTeX

Price:

Permissions CCC:

Request Permissions

Permissions PLS:

Request Permissions

Сopyright:

Citation:

* - Corresponding Author

References

[1] Fair Scheduler Guide[EB/OL]. http: /hadoop. apache. org/common/docs/current/fair_scheduler. html, (2012).

Google Scholar

[2] Capacity Scheduler Guide [EB/OL]. http: /hadoop. apache. org/common/docs/current/capacity_scheduler. htm, (2012).

Google Scholar

[3] ZAHARIA M, BORTHAKUR D, SARMA J S. Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduleing[C]. EuroSys'10: Proceedings of the 5th European conference on Computer systems, 2010: 265-278.

DOI: 10.1145/1755913.1755940

Google Scholar

[4] POLO J, CARRERA D, BECERRA Y. Performance driven task co-scheduling for mapreduce environments [C]. Network Operations and Management Symposium (NOMS), IEEE, 2010: 373-380.

DOI: 10.1109/noms.2010.5488494

Google Scholar

[5] Lingfen Meng. PC cluster job scheduling algorithm research [D]. Dongying: China University of Petroleum (east China), (2009).

Google Scholar

[6] SONALI A, SHASHANK P, MILIND B. Characterization of hadoop jobs using unsupervised learning[C]. 2nd IEEE International Conference on Cloud Computing Technology and Science (CloudCom2010), 2010: 748- 753.

DOI: 10.1109/cloudcom.2010.20

Google Scholar

[7] Jun Qin, Jianping Zhang, Hao Wang, JiaBin Wei. Graphs cluster scheduling strategy based on ant colony optimization [J]. Computer technology and development. 2013. 23(6): 74-78 ISSN 1673-629X.

Google Scholar

[8] Kai Wang. The research and implementation of multi-user scheduling method of MapReduce cluster [D]. Changsha: national university of defense technology, (2010).

Google Scholar