The Metadata Dynamic Load-Balancing Strategy of Distributed Filesystem Based on Hash Tags

Abstract:

With the development of information technology, distributed file system is widely used in massive information storage. Usually, distributed file system uses metadata server to achieve quick access to files according to directory, thus the organization and management of metadata are the keys to the file system performance. In general, directory subtree partition method and hash algorithm are used by existing mass storage system to manage metadata. However, to solve the problems, like low access efficiency of metadata, ineffective balance of load and poor extensibility, in existing metadata management strategy of distributed file system, dynamic load balance strategy of metadata based on hash tags is put forward, in which the tags act as partition granularity and hot tags of metadata will be hashed again to achieve the goal of load balance. The experimental results in this paper turn out that modified metadata management strategy based on hash tags has greater system throughput and less average response time than the one based on tags.,

• Periodical: Applied Mechanics and Materials (Volumes 556-562)
• Pages: 4009-4013
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.556-562.4009
• Online since: May 2014
• Authors: Yi Jiang*, Qiang Xiao, Rong Huang, An Ping Xiong
• Keywords: Distributed File System, Dynamic Load Balance, Hash Tags, Metadata

References

[1] Satyanarayanan M, Kistler J J, Kumar P, et al. Coda: A highly available file system for a distributed workstation environment[J]. Computers, IEEE Transactions on, 1990, 39(4): 447-459.

DOI: 10.1109/12.54838

[2] Rodeh O, Teperman A. zFS-a scalable distributed file system using object disks[C]/Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings. 20th IEEE/11th NASA Goddard Conference on. IEEE, 2003: 207-218.

DOI: 10.1109/mass.2003.1194858

[3] Xing J, Xiong J, Sun N, et al. Adaptive and scalable metadata management to support a trillion files[C]/Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis. ACM, 2009: 26.

DOI: 10.1145/1654059.1654086

[4] McKusick M K, Ganger G R. Soft Updates: A Technique for Eliminating Most Synchronous Writes in the Fast Filesystem[C]/USENIX Annual Technical Conference, FREENIX Track. 1999: 1-17.

[5] WANG Juan, FENG Dan, WANG Fang, et al． Load Balancing Algorithm in MetaData Server Cluster [J]. Journal of Chinese Computer Systems, 2009. 4.

[6] Karger D, Lehman E, Leighton T, et al. Consistent hashing and random trees: Distributed caching protocols for relieving hot spots on the World Wide Web[C]/Proceedings of the twenty-ninth annual ACM symposium on Theory of computing. ACM, 1997: 654-663.

DOI: 10.1145/258533.258660

[7] SHU Wan-neng, ZHENG Shi-jue, CHEN Guang-dong, et al. Research on Load Balanced Algorithm for Campus Grid [J]. COMPUTOR TECHNOLOGY AND DEVELOPMENT, 2006, 6(1).