A Universal Storage Architecture for Heterogeneous Data of Internet of Things

Abstract:

With the rapid development of the Internet of Things technology, IOT terminal equipments collect a large amount of data. So the preprocessing and storage become a big challenge. In this paper we presents a universal preprocessing and storage architecture for IOT data in cloud environment. In the data preprocessing module, with the sensor equipments are not stable, a large number of mistake, missing data will be produced, so we propose a imputation algorithm based on clustering to perform data preprocessing. For the data storage module, because of the existence of a large number of unstructured and semi-structured data, we present a storage architecture for heterogeneous data in cloud environment. Experiments show that our architecture can effectively complete the data preprocessing and storage, for the subsequent work, such as query data provides good support.

• Periodical: Applied Mechanics and Materials (Volumes 713-715)
• Pages: 2423-2426
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.713-715.2423
• Online since: January 2015
• Authors: Yong Lin Leng*, Qing Chen Zhang, Yue Wang
• Keywords: Heterogeneous Data, IOT, Preprocessing, Storage Architecture

References

[1] L. Atzori, A. Iera, and G. Morabito, The internet of things: a survey, Computer Networks, vol. 54, pp.2787-2805, (2010).

DOI: 10.1016/j.comnet.2010.05.010

[2] L.Y. Cui, S. Kumara, and R. Albert, Complex networks: An engineering view, Circuits and Systems Magazine, IEEE, vol. 10, no. 3, pp.10-25, (2010).

DOI: 10.1109/mcas.2010.937883

[3] R. Cattell, Scalable SQL and NoSQL data stores, , ACM SIGMOD Record, vol. 39, no. 4, pp.12-27, (2011).

DOI: 10.1145/1978915.1978919

[4] G. DeCandia, D. Hastorun, M. Jampani, et al, Dynamo: amazon's highly available key-value store, , Proceedings of twenty-first ACM SIGOPS symposium on Operating systems principles, vol. 14, no. 17, pp.205-220, (2007).

DOI: 10.1145/1294261.1294281

[5] A. Seović, M. Falco, P. Peralta, Oracle Coherence 3. 5, Packt Publishing Ltd, (2010).

[6] K. Chodorow,M. Dirolf, MongoDB: the definitive guide, " O, Reilly Media, (2010).

[7] C. Anderson,J. Lehnardt,N. Slater. CouchDB: The Definitive Guide: The Definitive Guide", O, Reilly Media, (2010).

[8] J.M. Jerez, I. Molina, P.J. García-Laencina, et al. Missing data imputation using statistical and machine learning methods in a real breast cancer problem, Artificial intelligence in medicine, vol. 50, no. 2, pp.105-115, (2010).

DOI: 10.1016/j.artmed.2010.05.002

[9] B. Twala, M.C. Jones, D.J. Hand, Good methods for coping with missing data in decision trees, , Pattern Recognition Letters, vol. 29, no. 7, pp.950-956, (2008).

DOI: 10.1016/j.patrec.2008.01.010

[10] Q.C. Zhang Z.K. Chen A.L. Lv, et al. A Universal Storage Architecture for Big Data in Cloud Environment, IEEE International Conference on GreenCom-iThings-CPSCom, pp.476-480, (2013).

[11] H. LI, E. Amani, P. LI, M. WU, Imputation algorithm of missing values based on EM and Bayesian network, Computer Engineering and Applications, vol. 46, pp.123-125, (2010).

[12] S. Wu,X.D. Feng Z.G. Shan, Missing Data Imputation Approach Based on Incomplete Data Clustering, , Chinese Journal of Computers, vol. 35, no. 8, pp: 1726-1738, (2012).

DOI: 10.3724/sp.j.1016.2012.01726