Unsupervised Mining of Visually and Temporally Consistent Shots for Sports Video Summarization

Abstract:

Video summarization aims at providing compact representation containing enough information for users to understand the entire content or important events, which serves as the fundamental process in content-based video analysis. This paper presents a novel sport video summarization algorithm by mining consistent field-of-views applied visual and temporal information in a totally unsurprised manner. After videos are broken into shots, a content-based similarity measure is proposed in the shot level to structurally analysis the visually matching cost of original videos. Then modified agglomerative hierarchical clustering is performed with an energy-based function to match the statistical distribution of various views in game videos and a refined distance metric is proposed as similarity measure of two shots. Extended temporal prior is introduced to meet the fact that temporally neighbored shots with similar duration are more likely to be in the same clusters. Experiments on a database of 6 sport genres with over 10251 minutes of videos from different sources achieved an average accuracy of 91.5% and quantitative results are presented to justify each choice made in the design of our algorithm. Our proposed algorithm is applied for the non-linear browsing service of Orangesports by France Telecomm and an android based app has been implemented for smart mobile devices.

• Periodical: Applied Mechanics and Materials (Volumes 284-287)
• Pages: 3140-3144
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.284-287.3140
• Online since: January 2013
• Authors: Yuan Dong, Ji Wei Zhang, Jian Zhao, Wei Liu, Kun Tao
• Keywords: Multimedia Content Analysis, Sport Video Analysis, Video Summarization

References

[1] Jiwei Zhang, Yuan Dong, Kun Tao, Xiaofu Chang, a modified sports genre categorization framework based on close-up view predetection. IC-BNMT(2011) Oct 301-305, Shenzhen, China.

DOI: 10.1109/icbnmt.2011.6155945

[2] M. Ellouze, N. Boujemaa, and A. Alimi, Im (s) 2: Inter-active movie summarization system. Journal of Visual Communication and Image Representation, 21(4): 283–294, (2010).

DOI: 10.1016/j.jvcir.2010.01.007

[3] Y. Gao and Q. Dai, Shot-based similarity measure for content-based video summarization. In 15th IEEE International Conference on Image Processin, pages 2512–2515, IEEE, (2008).

DOI: 10.1109/icip.2008.4712304

[4] L. Herranz and J. Martinez. a framework for scalable summarization of video. IEEE Transactions on Circuits and Systems for Video Technology, 20(9): 1265–1270, (2010).

DOI: 10.1109/tcsvt.2010.2057020

[5] C. Ngo, T. Pong, and H. Zhang. on clustering and retrieval of video shots through temporal slices analysis. IEEE Trans. on Multimedia, 4(4): 446–458, (2002).

DOI: 10.1109/tmm.2002.802022

[6] S. Baker, C. Zitnick, and G. Schroff, clustering videos by location, US Patent 12/416, 152, (2009).

[7] B. Chen, J. Wang, and J. Wang. A novel video summarization based on mining the story-structure and semantic relations among concept entities. IEEE Trans on Multimedia, 11(2): 295–312, (2009).

DOI: 10.1109/tmm.2008.2009703

[8] C. Siagian and L. Itti. Rapid biologically-inspired scene classification using features shared with visual attention. IEEE Trans on PAMI, 29(2): 300–312, (2007).

DOI: 10.1109/tpami.2007.40

[9] L. Najman and M. Couprie, building the component tree in quasi-linear time. IEEE Trans. on Image Processing, 15(11): 3531–3539, (2006).

DOI: 10.1109/tip.2006.877518

[10] Engin Mendi and Coskun Bayrak, Summarization of MPEG Compressed Video Sequences, Advanced. Science. Letter. 4, 3706-3708 (2011).

DOI: 10.1166/asl.2011.1892