DMFSsim: A Distributed Metadata File System Simulator

Abstract:

Parallel file systems deploy multiple metadata servers to distribute heavy metadata workload from clients. With the increasing number of metadata servers, metadata-intensive operations are facing some problems related with collaboration among them, compromising the performance gain. Consequently, a file system simulator is very helpful to try out some optimization ideas to solve these problems. In this paper, we propose DMFSsim to simulate the metadata-intensive operations on large-scale distributed metadata file systems. DMFSsim can flexibly replay traces of multiple metadata operations, support several commonly used metadata distribution algorithms, simulate file system tree hierarchy and underlying disk blocks management mechanism in real systems. Extensive simulations show that DMFSsim is capable of demonstrating the performance of metadata-intensive operations in distributed metadata file system.

• Periodical: Applied Mechanics and Materials (Volumes 241-244)
• Pages: 1556-1561
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.241-244.1556
• Online since: December 2012
• Authors: Qi Meng Wu, Ke Xie, Ming Fa Zhu, Li Min Xiao, Li Ruan
• Keywords: Distributed Metadata, File System, Metadata-Intensive Operation, Simulator

References

[1] P.H. Carns, W.B. Ligon, R.B. Ross and R. Thakur, PVFS: A parallel file system for linux clusters, Proceedings of the 4th annual Linux Showcase & Conference, Vol. 4 (2000), p.28.

[2] P. Schwan, Lustre: Building a file system for 1000-node clusters, Proceedings of the 2003 Linux Symposium (2003).

[3] Information on http: /www. top500. org.

[4] N. Ali, A. Devulapalli, D. Dalessandro, P. Wyckoff and P. Sadayappan, Revisiting the metadata architecture of parallel file systems, Petascale Data Storage Workshop (2008), p.1.

DOI: 10.1109/pdsw.2008.4811892

[5] Y. Zhu, H. Jiang, J. Wang and F. Xian, HBA: Distributed metadata management for large cluster-based storage systems Parallel and Distributed Systems, IEEE Transactions, Vol. 19(2008), p.750.

DOI: 10.1109/tpds.2007.70788

[6] S. R. Alam, H. N. El-Harake, K. Howard, N. Stringfellow and F. Verzelloni, Parallel I/O and the Metadata Wall, Proceedings of the sixth workshop on Parallel Data Storage (2011), p.13.

DOI: 10.1145/2159352.2159356

[7] S. Lang, P. Carns, R. Latham, R. Ross, K. Harms and W. Allcock, I/O performance challenges at leadership scale, Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis (2009), p.40.

DOI: 10.1145/1654059.1654100

[8] W. Yu, J. S. Vetter and H. S. Oral, Performance characterization and optimization of parallel I/O on the Cray XT, Parallel and Distributed Processing, IEEE International Symposium (2008), p.1.

DOI: 10.1109/ipdps.2008.4536277

[9] A. Devulapalli and P. W. Ohio, File creation strategies in a distributed metadata file system, IEEE Int. Conf. on Parallel and Distributed Processing Symposium (2007), p.1.

DOI: 10.1109/ipdps.2007.370295

[10] C. Biardzki and T. Ludwig, Analyzing metadata performance in distributed file systems, Parallel Computing Technologies (2009), p.8.

DOI: 10.1007/978-3-642-03275-2_2

[11] Y. Liu, R. Figueiredo, D. Clavijo, Y. Xu and M. Zhao, Towards Simulation of Parallel File System Scheduling Algorithms with PFSsim, Proceedings of the 7th IEEE International Workshop on Storage Network Architecture and Parallel I/O (2011).

[12] X. Li, B. Dong, L. Xiao, L. Ruan and D. Liu, CEFLS: A cost-effective file lookup service in a distributed metadata file system, The 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (2012).

DOI: 10.1109/ccgrid.2012.29

[13] E. Molina-Estolano, C. Maltzahn, J. Bent and S. A. Brandt, Building a parallel file system simulator, Journal of Physics: Conference Series, Vol. 180 (2009), p.12050.

DOI: 10.1088/1742-6596/180/1/012050

[14] P. H. Carns, B. W. Settlemyer and W. B. Ligon, Using server-to-server communication in parallel file systems to simplify consistency and improve performance, Proceedings of the 2008 ACM/IEEE conference on Supercomputing (2008), p.6.

DOI: 10.1109/sc.2008.5214724

[15] A. Varga, The OMNeT++ discrete event simulation system, Proceedings of the European Simulation Multiconference (2001), p.319.

[16] J. S. Bucy, J. Schindler, S. W. Schlosser and G. R. Ganger, The DiskSim simulation environment version 4. 0 reference manual, (2008).

[17] Information on http: /sourceforge. net/projects/tinyxml.