Metadata Indexing Sub-System for Distributed File System

De Jiao Niu; Tao Cai; Yong Zhao Zhan; Shi Guang Ju

doi:10.4028/www.scientific.net/AMM.143-144.864

Paper Titles

Application on OFDM-Based Transform Domain Communication System in Mine
p.843

Research and Implementation of the Optimal Design Pattern of Electronic Commerce Based on J2EE
p.849

Fast Routing System Research and Application Based on Pro-E
p.854

Authentication Protocol Security Assessment Framework Based on Attack Classification
p.859

Metadata Indexing Sub-System for Distributed File System
p.864

Application of Line-up Competition Algorithm with Variable Weight for Tree Pipe Network Layout Optimization
p.869

Study on Equipment Scheduling of ZPMC Automated Container Terminal Based on Improved PSO
p.879

Real-Time Customization and Motion Simulation of the Omnidirectional Wheelchair Based on VRML
p.885

Transformation of Double Housing Planer and Analysis of its Efficiency Increasing
p.890

HomeApplied Mechanics and MaterialsApplied Mechanics and Materials Vols. 143-144Metadata Indexing Sub-System for Distributed File...

Metadata Indexing Sub-System for Distributed File System

Abstract:

The efficiency of metadata indexing is important to the performance of distributed file system. Time and space spending of current metadata management algorithms are unstable. In this paper, we use B-tree to index the metadata of distributed file system. Lustre is an open source distributed file system in which Hash function is used to manage the metadata. We implement the prototype of metadata indexing sub-system on Lustre and use Iozone to test the I/O performance of Lustre with and without the metadata indexing sub-system respectively. The simulation results show that Lustre with the metadata indexing sub-system has higher adaptability than Lustre with Hash-based metadata management algorithm.

You might also be interested in these eBooks

View Preview

Info:

Periodical:

Applied Mechanics and Materials (Volumes 143-144)

Pages:

864-868

DOI:

https://doi.org/10.4028/www.scientific.net/AMM.143-144.864

Citation:

Cite this paper

Online since:

December 2011

Authors:

De Jiao Niu, Tao Cai, Yong Zhao Zhan, Shi Guang Ju

Keywords:

Distributed File System, Lustre, Metadata Indexing, Metadata Management

Export:

RIS, BibTeX

Price:

Permissions CCC:

Request Permissions

Permissions PLS:

Request Permissions

Сopyright:

Citation:

References

[1] D. Roselli, J. Lorch, and T. Anderson. A Comparison of File System Workloads. Proceedings of the 2000 USENIX Annual Technical Conference, page 41-54, June (2000).

Google Scholar

[2] M. Satyanarayanan, J.J. Kistler, P. Kumar, M.E. Okasaki, E.H. Siegel, and D.C. Steere. Coda: A Highly Available File System for Distributed Workstation Environments. IEEE Transaction on Computers, vol. 39, no. 4, Apr. (1990).

DOI: 10.1109/12.54838

Google Scholar

[3] E. Riedel, M. Kallahalla, and R. Swaminathan. A Framework for Evaluating Storage System Security. Proc. First Usenix Conference on File and Storage Technology, pp.15-30, Mar. (2002).

Google Scholar

[4] S.A. Weil, K.T. Pollack, S.A. Brandt, E.L. Miller. Dynamic Metadata Management for Petabyte-Scale File Systems. ACM/IEEE Conference on Supercomputing, DC, USA, 2004: 4.

DOI: 10.1109/sc.2004.22

Google Scholar

[5] Corbett P F, Feitelso D G. The Vesta parallel file system[J]. ACM Transaction on Computer System, 1996, 14(3): 225-264.

DOI: 10.1145/233557.233558

Google Scholar

[6] Brandt S A, Lan Xue, Miller. Efficient metadata management in large distributed file systems: Proceedings of the 20th IEEE/11th NASA Goddard Conference on Mass Storage System and Technologies. San Diego, 2003: 290-297.

DOI: 10.1109/mass.2003.1194865

Google Scholar

[7] Peter J. Braam. The Lustre Storage Architecture. Cluster File Systems, Inc. 2003. 10.

Google Scholar