Challenges of Cloud Computing Evaluation

Abstract:

Cloud computing is one of the hottest topics of IT research and industry because of its easy usage and flexible payment. At present, many researchers and projects focus on improving the efficiency and performance of cloud computing. Few researchers have recognized the importance the evaluation of cloud computing and started to investigate the evaluation methods. This paper first surveys research papers and technology reports about cloud computing evaluation. Then the characteristics and influential factors of cloud computing are analyzed. Some important unsolved issues of cloud computing evaluation are proposed. Finally, this paper gives some advice and points out future open problems of evaluation.

• Periodical: Applied Mechanics and Materials (Volumes 198-199)
• Pages: 396-401
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.198-199.396
• Online since: September 2012
• Authors: Peng Shi, Li Xin Zhang, Yi Bo Ai
• Keywords: Benchmark, Cloud Computing, Evaluation, Web Service

References

[1] R. Chellappa, Intermediaries in cloud-computing: A new computing paradigm, in INFORMS Dallas 1997 Cluster: Electronic Commerce, Dallas, Texas, (1997).

[2] Information on http: /aws. amazon. com.

[3] S. Lohr, Google and IBM join in cloud computing research, New York Times, (2007).

[4] D. Krissi, Distinguishing Cloud Computing from Utility Computing, Ebizq. net, information on http: /www. ebizq. net/blogs/saasweek/2008/03/distinguishing_cloud_computing.

[5] F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach Mike Burrows, et al. Bigtable: A Distributed Storage System for Structured Data, Proceeding of the 7th Symposium on Operating System Design and Implementation, Vol. 7, Seattle, WA, November, (2006).

DOI: 10.1145/1365815.1365816

[6] J. Dean and S. Ghemawat, MapReduce: Simplifed Data Processing on Large Clusters, Commun. ACM, vol. 51, pp.107-113, (2008).

DOI: 10.1145/1327452.1327492

[7] Information on http: /en. wikipedia. org/ wiki/Evaluation.

[8] J. J. Dongarra, P. Luszczek and A. Petitet, The LINPACK benchmark: Past, present, and future, Concurrency and Computation: Practice and Experience, 2003, vol. 15, p.1−18.

DOI: 10.1002/cpe.728

[9] D. Bailey, T. Harris, _ W. Saphir, R. Wijngaart, A. Woo and M. Yarrow, The NAS parallel benchmarks 2. 0, information on http: /www. nas. nasa. gov/assets/pdf/techreports/1995/nas-95-020. pdf.

[10] SPEC standards, information on http: /www. spec. org.

[11] E. Joseph, C. G. Willard, M. Swenson and D. Goldfarb, A new HPC technical computing benchmark: the IDC balanced rating, IDC Bulletin W IDC Bulletin #W24824, June, (2001).

[12] J. Dongarra and P. Luszczek, Introduction to the HPC challenge benchmark suite, ICL Technical Report, ICL-UT-05-01, (2005).

[13] J. A. McCall, P. K. Richards and G. F. Walters, Factors in software quality, RADC (Rome Air Development Centre): Technical Report RADC-TR-77-363, (1977).

[14] L. Zeng, Benat allam B. QoS aware middleware for Web service composition, IEEE Transactions on Software Engineering, 2004, 30(5), pp.311-327.

DOI: 10.1109/tse.2004.11

[15] B. Medjah ed, Semantic Web enabled composition of Web services, [ Ph. D. dissertation ]. Virginia Polyt echnic Institute and State University, Virginia, USA, (2004).

[16] W.J. Yang, J.Z. Li, K.H. Wang, Domain-Adaptive Service Evaluation Model, Chinese Journal of Computers, No. 4, Vol. 28, pp.514-523, (2005).

[17] G. Q. Liu, Z. L. Zhu, H. Wang, D. CH. Li and Y. Liu, Model for Evaluating QoS Trust worthiness of Web Service, Journal of Chinese Computer Systems, Vol. 30, No. 11, pp.2216-2221, (2009).

[18] Y. L. Liu, X. Y. Bai, G. Cheng and L. J. Wang, Policy-based runtime performance evaluation and validation of Web Service, ACTA ELECTRONIC ASINIC, vol. 32, no. 2A, pp.182-187, (2010).

[19] Information on http: /www. cjsdn. net/post/view?bid=2&id=4913&sty=3.

[20] J. Scanlon and B. Wieners. The Internet Cloud, Thestandard. com, information on http: /www. thestandard. com/article/0, 1902, 5466, 00. html. Retrieved 2010-08-22.

[21] B. Rochwerger, J. Caceres, R.S. Montero, D. Breitgand, E. Elmroth, et al. The RESERVOIR Model and Architecture for Open Federated Cloud Computing, IBM Journal of Research and Development, vol. 53, no. 4, (2009).

DOI: 10.1147/jrd.2009.5429058

[22] Ch. T. Chen and K. H. Lin, A decision-making method based on interval-valued fuzzy sets for cloud service evaluation, 4th International Conference on New Trends in Information Science and Service Science (NISS), pp.556-564, (2010).

[23] H. Kim, H. Lee, W. Kim and Y. Kim，A Trust Evaluation Model for Cloud Computing, Grid and Distributed Computing Communications in Computer and Information Science, vol. 63, pp.184-192, (2009).

DOI: 10.1007/978-3-642-10549-4_22

[24] J. Y. Lee, J. W. Lee, D. W. Cheun and S. D. Kim，A Quality Model for Evaluating Software-as-a-Service in Cloud Computing, 2009 Seventh ACIS International Conference on Software Engineering Research, Management and Applications，pp.261-266, (2009).

DOI: 10.1109/sera.2009.43