Mix-P2P Architecture of Distributed Storage System Based on HDFS

Abstract:

Aiming at the most popular Hadoop's master/slaves architecture in cloud computing, this paper presents new distributed storage architecture—MP2PDSA. Firstly, this architecture built a P2P network of SuperNodes without any master, which changed the master control of HDFS. Through the SuperNodes' collaboration, MP2PDSA enabled the storage system no longer rely on central control, which has effectively overcome Single-Point Failure, performance bottlenecks and so on. Secondly, in order to increase the availability of the whole storage system, through the coordination mechanism of P2P_SuperNodes, MP2PDSA achieved a rapid time to take over the failure subgroup without any manual intervention. Finally, MP2PDSA has a high scalability and availability compared with HDFS

• Periodical: Advanced Materials Research (Volume 382)
• Pages: 92-95
• DOI: https://doi.org/10.4028/www.scientific.net/AMR.382.92
• Online since: November 2011
• Authors: Zhan Jie Wang, Li Jun Zhang
• Keywords: Availability, Distributed Storage, MixP2P, Scalability

References

[1] HDFS. http: /hadoop. apache. org/hdfs (2011).

[2] Luis Bautista, Alain April. A Distributed Solution to Namenode Failure in the Hadoop Distributed File System (2010).

[3] Tian Jing, Dai Yafei. A Survey of Durable Peer-to-Peer Storage Techniques. Journal of Software (2007).

[4] Hou Mengshu. The Research on P2P-based distributed storage and its technologies. University of Electronic Science and Technology of China (2005).

[5] Song Wei. The Research of Nodes Autonomyn in Distributed Storage System, South China University of Technology (2010).