Deep Learning Approach to New Age Cinematic Video Editing

Abstract:

This paper gives a new technique for video editing by introducing modules that are trained to create realistic and dramatic sky backgrounds in videos. This project is different from what is being used in the video editing ecosystem as it does not require any static photos or inertial measurements. The module can be simply used on any device without having any prerequisites in the device. This is a game- changer when it comes to capturing cinematic sky videos. This project is further branched into three different modules to segregate the different tasks including sky routing, flow reckoner, and image emulsifier. These methods will run in real- time and are user friendly. This project can generate high fidelity videos with different lighting and dramatics in outdoor environments. Adding further we can also easily synthesize different weather conditions. This editing technique is much simpler and easier giving a more aesthetic image for cinematic shots.

• Periodical: Advances in Science and Technology (Volume 124)
• Pages: 250-258
• DOI: https://doi.org/10.4028/p-1ol72g
• Online since: February 2023
• Authors: Disha Dutta*, Shrunkhala Jawade, Syed Mateen Hussain
• Keywords: Background Enhancer, Background Replacement, Cinematic Sky Lightning Adjustment, Deep Learning, Sky Background, Video Editing, Weather Change

References

[1] Yanting Pei, Yaping Huang, Qi Zou, Xingyuan Zhang, Song Wang, Effects of Image Degradation and Degradation Removal to CNN-based Image Classification,, Volume: 43, Issue: 4, November (2019).

[2] Xinguang Xiang , Hao Wei, and Jinshan Pan, Member, IEEE, Deep Video Deblurring Using Sharpness Features From Exemplars,, Volume: 29, September (2020).

DOI: 10.1109/tip.2020.3023534

[3] Jun Chu, Xuji Tu, Lu Leng, Jun Miao, Double-Channel Object Tracking With Position Deviation Suppression,, Volume: 8, December (2019).

DOI: 10.1109/access.2019.2961778

[4] Wenguan Wang, Jianbing Shen, Xiankai Lu, Steven C. H. Hoi, Haibin Ling, Paying Attention to Video Object Pattern Understanding,, Volume: 43, Issue: 7, January (2020).

[5] Tao Zhuo, Zhiyong Cheng, Peng Zhang, Yongkang Wong and Mohan Kankanhalli, Unsupervised Online Video Object Segmentation with Motion Property Understanding,, Volume: 29, July (2019).

DOI: 10.1109/tip.2019.2930152

[6] Tingtian Li, Y.H. Chan and Daniel P.K. Lun, 2021, Improved Multiple-Image Based Reflection Removal Algorithm Using Deep Neural Networks,, Volume 30, October (2020).

DOI: 10.1109/tip.2020.3031184

[7] S Ghosh, S P Pathak, K Kabasi, T K Moharana, M Bagchi, N Dasgupta, S Chaudhari and S Kumar, Single Tone Background Removal Applying Dual Threshold and Dynamic Clustering,, April (2019).

DOI: 10.1109/iccsp.2019.8698078

[8] He Zhang, Jianming Zhang, Federico Perazzi, Zhe Lin, Vishal M. Patel, Deep Image Compositing,, June (2021).

[9] Omar Elharrouss, Noor Al-Maadeed, Somaya Al-Maadeed, Video Summarization based on Motion Detection for Surveillance Systems,, July (2019).

DOI: 10.1109/iwcmc.2019.8766541

[10] Xiao Wen, Haoran Zhang,Video Target Detection Algorithm Based on Cuckoo Search and Gaussian Background Mixture Model,, May (2020).

DOI: 10.1109/eitce47263.2019.9095107

[11] Zhao, Minda, Ling, Qiang, PWStableNet: Learning Pixel-Wise Warping Maps for Video Stabilization,, Volume 29 January (2020).

DOI: 10.1109/tip.2019.2963380

[12] Wang, Zhongqiang, Zhang, Lei, Huang, Hua, High Quality Real-time Video Stabilization Using Trajectory Smoothing and Mesh-based Warping,, Volume 6, April (2018).

DOI: 10.1109/access.2018.2828653

[13] Fang, Ming, Wang, Yichen,Li, Hongna. Xu, Jing, Detail Maintained Low-Light Video Image Enhancement Algorithm,, October (2018).

DOI: 10.1109/icma.2018.8484390

[14] Kolaman, Amir, Malowany, Dan, Hagege, Rami R., Guterman, Hugo, Light Invariant Video Imaging for Improved Performance of Convolution Neural Networks,, Volume 29, Issue 6, June (2019).

DOI: 10.1109/tcsvt.2018.2846695

[15] Huang, Hua, Wei, Xiao-Xiang, Zhang, Lei, Encoding Shaky Videos By Integrating Efficient Video Stabilization,, Volume 29, Issue 5, May (2019).

[16] https://en.wikipedia.org/wiki/Kernel_density_estimation.

[17] https://groups.csail.mit.edu/vision/datasets/ADE20K.