Automated Random Testing of Image Processing Applications with Effectiveness Measure

Abstract:

Automating software testing process has remarkably reduced the cost of manual efforts and made testing considerably fast. Recently, image processing has been extensively used in different software applications and their significance implies a well-tested reliable system. Testing of these applications require semantically complex multidimensional image data. Whereas, the image databases are extremely large in size and testing of application on each image is not possible in constrained time. In order to meet these challenges, an automated testing tool is required which is capable of reporting bugs using a finite number of input images and can quantify its effectiveness on the basis of a coverage metric. In this paper, we present a tool IMTEST based on a novel technique to test image processing applications that aims to achieve high program coverage and capable to report bugs that manifest during testing process. It selects images randomly from an image database and gives input to the instrumented version of the code under test. The instrumented code replica is generated by our tool, inserting additional analysis code apart from original code. This allows us to measure the coverage achieved during program execution for a particular input image. A coverage vector is designed to record program coverage during iterative execution of instrumented program under test for different input images. Any unexpected program behavior is recorded as bug that manifests during testing process. A stopping criteria is used to conclude the testing in a considerable time duration. The tool is tested on real applications and the result shows that IMTEST reported real bugs which are reproducible with the specified images and is useful to improve software quality.

• Periodical: Applied Mechanics and Materials (Volume 555)
• Pages: 787-795
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.555.787
• Online since: June 2014
• Authors: Tahir Jameel*, Meng Xiang Lin, Saqib Yousaf
• Keywords: Input Generation, Random Testing, Software Testing

References

[1] Cai, Xia, and Michael R. Lyu. The effect of code coverage on fault detection under different testing profiles., ACM SIGSOFT Software Engineering Notes. Vol. 30. No. 4. ACM, (2005).

DOI: 10.1145/1082983.1083288

[2] Whittaker, James A. What is software testing? And why is it so hard?., Software, IEEE 17. 1 (2000): 70-79.

DOI: 10.1109/52.819971

[3] Mayer, Johannes. On Testing Image Processing Applications with Statistical Methods., Software Engineering. (2005).

[4] Godefroid, Patrice, Nils Klarlund, and Koushik Sen. DART: directed automated random testing., ACM Sigplan Notices. Vol. 40. No. 6. ACM, (2005).

DOI: 10.1145/1064978.1065036

[5] Itkonen, Juha, Mika V. Mantyla, and Casper Lassenius. How do testers do it? An exploratory study on manual testing practices., Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement. IEEE Computer Society, (2009).

DOI: 10.1109/esem.2009.5314240

[6] Pham, Hoang, and Xuemei Zhang. NHPP software reliability and cost models with testing coverage., European Journal of Operational Research 145. 2 (2003): 443-454.

DOI: 10.1016/s0377-2217(02)00181-9

[7] Wang, Zhou, Alan C. Bovik, and Ligang Lu. Why is image quality assessment so difficult?., Acoustics, Speech, and Signal Processing (ICASSP), 2002 IEEE International Conference on. Vol. 4. IEEE, (2002).

DOI: 10.1109/icassp.2002.1004620

[8] Kuhn, D. Richard, and Vadim Okun. Pseudo-exhaustive testing for software. " Software Engineering Workshop, 2006. SEW, 06. 30th Annual IEEE/NASA. IEEE, (2006).

DOI: 10.1109/sew.2006.26

[9] Knight, John C., Kevin G. Wika, and Shannon Wrege. Exhaustive Testing as a Verification Technique., Submitted to the International Symposium on Software Testing and Analysis. 1996. J. Clerk Maxwell, A Treatise on Electricity and Magnetism, 3rd ed., vol. 2. Oxford: Clarendon, 1892, pp.68-73.

[10] Ciupa, Ilinca, et al. Finding faults: Manual testing vs. random testing vs. user reports., SoftwareReliability Engineering, 2008. ISSRE 2008. 19th International Symposium on. IEEE, (2008).

DOI: 10.1109/issre.2008.18

[11] http: /www. mathworks. com/matlabcentral/fileexchange.

[12] Horgan, Joseph R., Saul London, and Michael R. Lyu. Achieving software quality with testing coverage measures., Computer 27. 9 (1994): 60-69.

DOI: 10.1109/2.312032

[13] Beizer, B., 1990. Software Testing Techniques, second ed. International.

[14] Coppit, David, et al. Software assurance by bounded exhaustive testing., Software Engineering, IEEE Transactions on 31. 4 (2005): 328-339.

DOI: 10.1109/tse.2005.52

[15] Jagannath, Vilas, et al. Reducing the costs of bounded-exhaustive testing., Fundamental Approaches to Software Engineering. Springer Berlin Heidelberg, 2009. 171-185.

DOI: 10.1007/978-3-642-00593-0_12

[16] Malaiya, Yashwant K., et al. Software reliability growth with test coverage., Reliability, IEEE Transactions on 51. 4 (2002): 420-426.

DOI: 10.1109/tr.2002.804489