High Performance Data Processing Pipeline of Chinese Solar Radio Heliograph

Abstract:

The Chinese Solar Radio Heliograph (CSRH) is a new generation radio heliograph would produce more than 4 terabytes data every day. As a aperture synthesis telescope, CSRH is facing the challenge of processing and storing such a vast data. Pipeline system is the key issue of data automatical processing for CSRH. In this study, to push the development of CSRH, we present a framework of high performance data processing pipeline system for saving and processing real-time observation data. The related techniques of pipeline software are presented in detail including raw data acquisition, UVFITS file, data calibration, parallel computing and data publication. The pipeline has been deployed and has played an important role for the development of CSRH.

• Periodical: Applied Mechanics and Materials (Volumes 347-350)
• Pages: 1012-1017
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.347-350.1012
• Online since: August 2013
• Authors: Fan Zhang, Feng Wang, Wei Wang, Wei Dai, Hui Deng, Kai Fan Ji, Yi Hua Yan
• Keywords: Data Processing, HPC

References

[1] Aschwanden and M. J, 2D feature recognition and 3D reconstruction in solar EUV images, , Solar Physics, 2005, 228, (1), pp.339-358.

DOI: 10.1007/s11207-005-2788-5

[2] Yan. Y, Zhang. J, Chen. Z, Wang. W, Liu. F, and Geng. L, Progress on Chinese Spectral Radioheliograph—CSRH construction, Book Progress on Chinese Spectral Radioheliograph—CSRH construction, (IEEE, 2011, edn. ), pp.1-4.

DOI: 10.1109/ursigass.2011.6051218

[3] Yan. Y, Zhang J, Wang. W, Liu. F, Chen. Z, and Ji. G, The Chinese Spectral Radioheliograph—CSRH, Earth Moon and Planets, 2009, 104, (1), pp.97-100.

DOI: 10.1007/s11038-008-9254-y

[4] Thompson. A. R, Moran. J. M, and Swenson Jr, G. W, Interferometry and synthesis in radio astronomy, Wiley-Vch, (2008).

[5] Wells. D, Greisen. E, and Harten. R, Fits-a flexible image transport system, Astronomy and Astrophysics Supplement Series, 1981, 44, p.363.

[6] Greisen. E, and Harten. R, An extension of FITS for groups of small arrays of data, Astronomy and Astrophysics Supplement Series, 1981, 44, p.371.

[7] Högbom. J, Aperture synthesis with a non-regular distribution of interferometer baselines, Astronomy and Astrophysics Supplement Series, 1974, 15, p.417.

[8] Clark. B, An efficient implementation of the algorithm'CLEAN', Astronomy and Astrophysics, 1980, 89, p.377.

[9] Cornwell. T, A method of stabilizing the clean algorithm, Astronomy and Astrophysics, 1983, 121, pp.281-285.

[10] Steer. D, Dewdney. P, and Ito. M, Enhancements to the deconvolution algorithm'CLEAN', Astronomy and Astrophysics, 1984, 137, pp.159-165.

[11] Ables. J, Maximum entropy spectral analysis, Astronomy and Astrophysics Supplement Series, 1974, 15, p.383.

[12] Gull. S, and Daniell. G, Image reconstruction from incomplete and noisy data, (1978).

[13] Ramesh. N, and Nityananda. R, Maximum entropy image restoration in astronomy, Annual review of astronomy and astrophysics, 1986, 24, pp.127-170.

DOI: 10.1146/annurev.aa.24.090186.001015

[14] Freeland. S, and Handy. B, Data analysis with the SolarSoft system, Solar Physics, 1998, 182, (2), pp.497-500.