A Reliability Enhance Method towards Hadoop Architecture

Abstract:

1PB or more of the data has been the concern of the reliability of the data in large data warehouses. It strongly depends on many interdependent system parameters, such as the replica placement policy, the number of nodes and Previous work has substantially and respectively discussed the individual effects of these parameters, and rarely provide its optimum value, there is no mention of their optimal combination. In this paper, we propose a new object-based repair Markov model. The analysis of this model based on three popular replica placement strategy, we calculate the personal best value of these parameters in the first, then work out the best combination of GA. Compared with the existing models, our model is easier to solve, and to achieve a more comprehensive and practical conclusions.

• Periodical: Applied Mechanics and Materials (Volumes 278-280)
• Pages: 1783-1786
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.278-280.1783
• Online since: January 2013
• Authors: Jia Jia Miao, Guo You Chen, Kai Du, Zhong Jun Fang
• Keywords: Data Reliability, Large-Scale Data Warehouse, Reliability Model

References

[1] http: /www. businessintelligencelowdown. com/2007/02/top_10_largest_. html. (2008).

[2] Reference information: The next wave, The Enterprise Storage Group, June (2002).

[3] Q. Lian, W. Chen, Z. Zhang, On the Impact of Replica Placement to the Reliability of Distributed Brick Sto-rage Systems, in ICDCS (2005).

DOI: 10.1109/icdcs.2005.56

[4] Ming Chen, Wei Chen, Likun Liu, and Zheng Zhang. An Analytical Framework and Its Applications for Studying Brick Storage Reliability. In SRDS (2007).

DOI: 10.1109/srds.2007.21

[5] S. Ghemawat, H. Gobioff, and S. -T. Leung, The Google File System, in Proc. of the 19th SOSP, (2003).

[6] ADYA, etc. FARSITE: Federated, Available, and Reliable Storage for an Incompletely Trusted Environment. In OSDI (2002).

DOI: 10.1145/1060289.1060291

[7] SANTOS, J., MUNTZ, R., AND RIBEIRO-NETO, B. Comparing Random Data Allocation and Data Striping in Multimedia Servers. In ACM SIGMETRICS (2000).

DOI: 10.1145/345063.339352

[8] H. Yu, P. B. Gibbons, and S. Nath, Availability of Multi-Object Operations", in Proc. of NSDI, 06.

[9] PATTERSON, D. A., GIBSON, G., AND KATZ, R. H. A Case for Redundant Arrays of Inexpensive Disks (RAID). In ACM SIGMOD (1988).

DOI: 10.1145/971701.50214

[10] KISTLER, J. J., AND SATYANARAYANAN, M. Disconnected Operation in the Coda File System. ACM Transactions on Computer Systems (Feb. 1992).

DOI: 10.1145/146941.146942

[11] OysteinTorbjornsen. Multi-Site Declustering Strategies for Very High Database Service Availability. Phd thesis University of Trondheim, Norway. (1995).