Metadata Caching Subsystem for Cloud Storage

Abstract:

Cloud storage is a hot topic in current research. Different from previous work, we emphasize the importance of metadata cache in the study of cloud storage. Because the efficiency of distributed file system has much effect on cloud storage The metadata operation accounts for more than 50% of the total file operation. So the strategy of efficient metadata management is important. There are three parts in this paper. We start with a brief introduction of cloud storage. Then a metadata caching algorithm for cloud storage is proposed. An additional discussion of its performance is also provided. The prototype which incorporates the proposed metadata caching algorithm is realized on Luster to evaluate its performance. Comparing experimental results from this study conclude that the metadata caching subsystem can improve the performance of cloud storage.

• Periodical: Applied Mechanics and Materials (Volume 214)
• Pages: 584-590
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.214.584
• Online since: November 2012
• Authors: De Jiao Niu, Tao Cai, Yong Zhao Zhan, Shi Guang Ju
• Keywords: Cloud Storage, Metadata Caching, Metadata Management Algorithm

References

[1] D.Roselli, J.Lorch, and T.Anderson. A Comparison of File System Workloads. Proceedings of the 2000 USENIX Annual Technical Conference, page 41-54, June 2000.

[2] M. Satyanarayanan, J.J. Kistler, P. Kumar, M.E. Okasaki, E.H. Siegel, and D.C. Steere. Coda: A Highly Available File System for Distributed Workstation Environments. IEEE Transaction on Computers, vol. 39, no. 4, Apr. 1990.

DOI: 10.1109/12.54838

[3] E. Riedel, M. Kallahalla, and R. Swaminathan. A Framework for Evaluating Storage System Security. Proc. First Usenix Conference on File and Storage Technology, pp.15-30, Mar. 2002.

[4] S.A. Weil, K.T. Pollack, S.A. Brandt, E.L. Miller. Dynamic Metadata Management for Petabyte-Scale File Systems. ACM/IEEE Conference on Supercomputing, DC, USA, 2004:4

DOI: 10.1109/sc.2004.22

[5] Brandt S A, Lan Xue, Miller. Efficient metadata management in large distributed file systems: Proceedings of the 20th IEEE/11th NASA Goddard Conference on Mass Storage System and Technologies. San Diego, 2003:290-297.

DOI: 10.1109/mass.2003.1194865

[6] Weijia Li, Wei Xue, Jiwu Shu, Weimin Zheng. Dynamic Hashing: Adaptive Metadata Management for Petabyte-scale File Systems. The IEEE Conference On Mass Storage Systems and Technologies, Maryland, USA, May, 2006.

[7] Yifeng Zhu, Hong Jiang, Jun Wang, et al. HBA: Distributed Metadata Management for Large Cluster-Based Storage Systems. IEEE Transaction on Parallel and Distributed Systems , 2008, 19(6):750-763.

DOI: 10.1109/tpds.2007.70788

[8] Andrew Leung, Minglong Shao, Timothy Bisson, Shankar Pasupathy, Ethan L. Miller, Spyglass: Fast, Scalable Metadata Search for Large-Scale Storage Systems, Proceedings of the 7th USENIX Conference on File and Storage Technologies, February 2009:153-166.

[9] Mu Fei, Xue Wei, Shu Jiwu, Zhen Weimin. Metadata management mechanism based on route directory. Journal of Tsinghua University, 2009, Vol 49, No.8:1229-1232

[10] Peter J. Braam. The Lustre Storage Architecture. Cluster File Systems, Inc. 2003.10