Recognition and Description of Unknown Everyday Objects by Using an Image Based Meta-Search Engine for Service Robots


Article Preview

In future applications service robots will operate in complex and unstructured environments. While performing daily tasks such systems will have to recognize many different unknown objects in order to be able to conduct continuative processes. To support enhanced functionalities of a service robot system an image based meta-search engine that is accessible for a machine can be used. In this contribution the realization of an approach to determine reasonable identification results for single and multiple everyday objects by using the Google reverse image search system is described. In order to deliver convenient results for the communication of a robotic system with a human user the generated search results are post-processed and prepared in order to provide reasonable object descriptions. To enable an intuitive application, control a gesture based approach allows potential users to initialize a search procedure by pointing at an object of interest while being observed by a camera system. The performed experiments show a quite reliable detection behavior for a reasonably broad variety of everyday objects.



Edited by:

Jörg Franke and Markus Michl




S. Reitelshöfer et al., "Recognition and Description of Unknown Everyday Objects by Using an Image Based Meta-Search Engine for Service Robots", Advanced Engineering Forum, Vol. 19, pp. 132-138, 2016

Online since:

October 2016




* - Corresponding Author

[1] IFR - The International Federation of Robotics, World Robotics 2014 Service Robots, (2014).

[2] B. Gates, A Robot in Every Home, Sci Am, vol. 296, no. 1, p.58–65, (2007).

[3] M. Hägele, N. Blümlein, and O. Kleinle, EFFIROB - Wirtschaftlichkeitsanalysen neuartiger Servicerobotik-Anwendungen und ihre Bedeutung für die Robotik-Entwicklung, Fraunhofer IPA, München, Feb. (2011).

[4] C. Ramer, J. Seßner, M. Scholz, Z. Xu, and J. Franke, Fusing low-cost sensor data for localization and mapping of automated guided vehicle fleets in indoor applications, in Proceedings of the 2015 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), p.65–70, (2015).


[5] J. Canny, A Computational Approach to Edge Detection, IEEE Trans. Pattern Anal. Mach. Intell, vol. PAMI-8, no. 6, p.679–698, (1986).


[6] J. G. Proakis, Dimitris G. Manolakis Digital Signal Processing: Englewood Cliffs, NJ: Prentice-Hall, (1996).

[7] D. Chen et al, Robust image retrieval using multiview scalable vocabulary trees, in IS&T/SPIE Electronic Imaging: SPIE, 72570V, (2009).

[8] S. Leutenegger, M. Chli, and R. Y. Siegwart, BRISK: Binary Robust invariant scalable keypoints, in 2011 IEEE International Conference on Computer Vision (ICCV), p.2548–2555, (2011).


[9] P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan, Object detection with discriminatively trained part-based models, (eng), IEEE transactions on pattern analysis and machine intelligence, vol. 32, no. 9, p.1627–1645, (2010).


[10] K. Goldberg and B. Kehoe, Cloud Robotics and Automation: A Survey of Related Work: Technical Report No. UCB/EECS-2013-5, Berkeley, Jan. (2013).

[11] B. Kehoe, A. Matsukawa, S. Candido, J. Kuffner, and K. Goldberg, Cloud-based robot grasping with the google object recognition engine, in 2013 IEEE International Conference on Robotics and Automation (ICRA), p.4263–4270, (2013).


[12] M. Jiang and L. Zhang, Big Data Analytics as a Service for Affective Humanoid Service Robots, Procedia Computer Science, vol. 53, p.141–148, (2015).


[13] K. Lai and D. Fox, Object Recognition in 3D Point Clouds Using Web Data and Domain Adaptation, The International Journal of Robotics Research, vol. 29, no. 8, p.1019–1037, (2010).


[14] C. R. Rao, The Use and Interpretation of Principal Component Analysis in Applied Research, Sankhyā: The Indian Journal of Statistics, Series A (1961-2002), vol. 26, no. 4, p.329–358, (1964).

[15] R. Lienhart, A. Kuranov, and V. Pisarevsky, Empirical Analysis of Detection Cascades of Boosted Classifiers for Rapid Object Detection, in Lecture Notes in Computer Science, Pattern Recognition, G. Goos, J. Hartmanis, J. van Leeuwen, B. Michaelis, and G. Krell, Eds, Berlin, Heidelberg: Springer Berlin Heidelberg, p.297–304, (2003).


[16] P. Viola and M. Jones, Rapid object detection using a boosted cascade of simple features, in 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, I-511-I-518, (2001).