Format Optimization for Periodical Snapshot Output in CFD Simulation

Abstract:

Computational Fluid Dynamics (CFD) simulation often needs to periodically output the intermediate results to files in the form of snapshots for visualization or restart, which seriously impacts the performance. Traditional CFD simulation takes ASCII format with typesetting in the snapshot output process, which results in precision loss of the data in the snapshot array and more storage space occupancy than the original binary format. In this paper, we propose the binary non-typesetting format optimization for the snapshot output in CFD simulation, aiming at eliminating the impact of ASCII format. In our optimization method, the snapshot array is output in binary format without typesetting. On one hand, it reserves the original binary precision in the output files, while the ASCII format always has precision loss, which seriously impacts the accuracy of CFD simulation; On the other hand, non-typesetting provide the opportunity of output the snapshot as a continuous memory block, which will signally improve the efficiency of the output. We design the multiply output mode framework for CFD applications through the combination of ASCII typesetting format and binary non-typesetting format. We implement this multiply output mode framework in the open source CFD software OpenFOAM. Experimental results on Tianhe-2 supercomputer demonstrate that our binary non-typesetting format optimization technique can achieve good optimization effect for the periodical snapshot output in CFD application.

• Periodical: Applied Mechanics and Materials (Volumes 457-458)
• Pages: 910-918
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.457-458.910
• Online since: October 2013
• Authors: Xiao Guang Ren, Yu Hua Tang, Xin Hai Xu
• Keywords: CFD, Format, Optimization, Output

References

[1] Mavriplis D J, Pirzadeh S. Large-scale parallel unstructured mesh computations for three-dimensional high-lift analysis[J]. Journal of aircraft, 1999, 36(6): 987-998.

DOI: 10.2514/2.2540

[2] Gropp W D, Kaushik D K, Keyes D E, et al. High-performance parallel implicit CFD[J]. Parallel Computing, 2001, 27(4): 337-362.

DOI: 10.1016/s0167-8191(00)00075-2

[3] Körner C, Pohl T, Rüde U, et al. Parallel lattice Boltzmann methods for CFD applications[M]/Numerical Solution of Partial Differential Equations on Parallel Computers. Springer Berlin Heidelberg, 2006: 439-466.

DOI: 10.1007/3-540-31619-1_13

[4] Aumann P, Barnewitz H, Schwarten H, et al. MEGAFLOW: Parallel complete aircraft CFD[J]. Parallel Computing, 2001, 27(4): 415-440.

DOI: 10.1016/s0167-8191(00)00068-5

[5] Xia B, Sun D W. Applications of computational fluid dynamics (CFD) in the food industry: a review[J]. Computers and Electronics in Agriculture, 2002, 34(1): 5-24.

DOI: 10.1016/s0168-1699(01)00177-6

[6] Ma X, Winslett M, Lee J, et al. Improving MPI-IO output performance with active buffering plus threads[C]/Parallel and Distributed Processing Symposium, 2003. Proceedings. International. IEEE, 2003: 10 pp.

DOI: 10.1109/ipdps.2003.1213165

[7] Ma X, Lee J, Winslett M. High-level buffering for hiding periodic output cost in scientific simulations[J]. Parallel and Distributed Systems, IEEE Transactions on, 2006, 17(3): 193-204.

DOI: 10.1109/tpds.2006.36

[8] No J, Thakur R, Choudhary A. High-performance scientific data management system[J]. Journal of Parallel and Distributed Computing, 2003, 63(4): 434-447.

DOI: 10.1016/s0743-7315(03)00036-4

[9] Mickler H, Knüpfer A, Kluge M, et al. Trace-based analysis and optimization for the Semtex CFD application–hidden remote memory accesses and I/O performance[C]/Euro-Par 2008 Workshops-Parallel Processing. Springer Berlin Heidelberg, 2009: 295-304.

DOI: 10.1007/978-3-642-00955-6_34

[10] Matsuno K, Ecer A, Periaux J, et al. HIGH-SPEED MASS STORAGE SYSTEM OF NUMERICAL SIMULATOR III AND IT'S BASIC I/O PERFORMANCE BENCHMARK[J]. Parallel Computational Fluid Dynamics 2002: New Frontiers and Multi-Disciplinary Applications, 2003: 157.

DOI: 10.1016/b978-044450680-1/50020-6

[11] Bordawekar R, del Rosario J M, Choudhary A. Design and evaluation of primitives for parallel I/O[C]/Proceedings of the 1993 ACM/IEEE conference on Supercomputing. ACM, 1993: 452-461.

DOI: 10.1145/169627.169782

[12] Kotz D. Disk-directed I/O for MIMD multiprocessors[J]. ACM Transactions on Computer Systems (TOCS), 1997, 15(1): 41-74.

DOI: 10.1145/244764.244766

[13] Moore J A, Quinn M J. Enhancing disk-directed I/O for fine-grained redistribution of file data[J]. Parallel Computing, 1997, 23(4): 477-499.

DOI: 10.1016/s0167-8191(97)00010-0

[14] Oldfield R A, Womble D E, Ober C C. Efficient parallel I/O in seismic imaging[J]. International Journal of High Performance Computing Applications, 1998, 12(3): 333-344.

DOI: 10.1177/109434209801200303

[15] http: /www. top500. org.