Design of BOOKKEEPING Subsystem of LHAASO Project

Abstract:

During the research of cosmic rays in high energy physics, the data produced by real-time observation and simulation experiments is increased all the time. Besides, various physics detector must have description to explain their characteristics. These data become great pressure for the research center. Once LHAASO project officially begins, the data it produced will be approximately several PB. How to store and get data with high rate, along with manage files efficiently, is an important issue.The topic is designing a data management system of LHAASO project. The design will be reference for the engineering of BOOKKEEPING in order to render excellent services for LHAASO project. Its key points are designing friendly and succinct page, using Hadoop platform to distributed store and manage data files, recommending data warehouse principle to associate data source, and uniting Kettle to transform different data files.

• Periodical: Applied Mechanics and Materials (Volumes 738-739)
• Pages: 1343-1348
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.738-739.1343
• Online since: March 2015
• Authors: Bei Bei Li, Jiang Xing Yu, Bing Suo, Hong He
• Keywords: Data Warehouse, Hadoop, Heterogeneous Database, High Energy Physics, KETTLE, LHAASO

References

[1] Y. M. Wang, G. H. Feng, Y. Xue, Summary of Hadoop Research in Recent Years in Foreign Countries, Computer Systems and Applications, vol. 22, no. 6, pp.1-5, (2013).

[2] D. R. Shen, G. Yu, X. T. Wang, T. Z. Nie and Y. Kou, Survey on NoSQL for management of big data, Journal of Software, vol. 24, no. 8, pp.1786-1803, May. (2013).

DOI: 10.3724/sp.j.1001.2013.04416

[3] Y. He, W. Q. Wang and F. Xue, Study of Massive Data Mining Based on Cloud Computing, Computer technology and development, vol. 23, no. 2, pp.69-72, Feb. (2013).

[4] B. Huang, S. R. Xu and W. Pu, Design and implementation of MapReduce-based data mining platform, Computer engineering and design, vol. 34, no. 2, Feb. (2013).

[5] D. W. Sun, G. R. Chang, S. Gao, L. Z. Jin and X. W. Wang, Modeling a Dynamic Data Replication Strategy to Increase System Availability in Cloud Computing Environments, Journal of computer science and technology, vol. 27, no. 2, Mar. (2012).

DOI: 10.1007/s11390-012-1221-4

[6] X. P. Qin, H. J. Wang, X. Y. Du and S. Wang, Big Data Analysis—Competition and Symbiosis of RDBMS and MapReduce, Journal of Software, vol. 23, no. 1, pp.32-45, (2012).

DOI: 10.3724/sp.j.1001.2012.04091

[7] S. Zeng, F. Z. Qi and M. Wang, Study and Implementation of Data Transfer System in Experiment of High Energy Physics, Computer Science, vol. 39, no. 6A, pp.93-95, June. (2012).

[8] Y. S. Xu, D. S. Zang and G. X. Sun, Design of a distributed file metadata management system, Computer Engineering and Application, vol. 48, no. 7, pp.1-4, Dec. (2012).

[9] W. Xiong and Z. B. Yu. A Characterization and Analysis of Distributed File Systems, Journal of Integration Technology, vol. 4, no. 1, pp.103-109, Nov. (2012).

[10] Z. Cao. A future project at tibet: the large high altitude air shower observatory (LHAASO),. Chinese Physics. Vol. 34, no. 2, pp.249-252, Feb. (2010).

DOI: 10.1088/1674-1137/34/2/018

[11] Y. D. Cheng and A. G. Liu. Research on the Key Technology of Data Management of High Energy Physics Grid, Application Research on Computers. vol. 24, no. 10, pp.1022-1029, Oct. (2007).

[12] C. Stanescu, Data Collection and Processing for ARGO Yanbajing Experiment, International Conference on Computing in High Energy and Nuclear Physics, Beijing, pp.825-836. (2001).