Video smoke removal based on low-rank tensor completion via spatial-temporal continuity constraint
Hu Zhu
Jiangsu Province Key Lab on Image Processing and Image Communication, Nanjing University of Posts and Telecommunication, Nanjing, China
Search for more papers by this authorGuoxia Xu
Department of Computer Science, Norwegian University of Science and Technology, Gjovik, Norway
Search for more papers by this authorLu Liu
Jiangsu Province Key Lab on Image Processing and Image Communication, Nanjing University of Posts and Telecommunication, Nanjing, China
Search for more papers by this authorCorresponding Author
Lizhen Deng
National Engineering Research Center of Communication and Network Technology, Nanjing University of Posts and Telecommunication, Nanjing, China
Correspondence
Lizhen Deng, National Engineering Research Center of Communication and Network Technology, Nanjing University of Posts and Telecommunication, Nanjing, China.
Email: [email protected]
Search for more papers by this authorHu Zhu
Jiangsu Province Key Lab on Image Processing and Image Communication, Nanjing University of Posts and Telecommunication, Nanjing, China
Search for more papers by this authorGuoxia Xu
Department of Computer Science, Norwegian University of Science and Technology, Gjovik, Norway
Search for more papers by this authorLu Liu
Jiangsu Province Key Lab on Image Processing and Image Communication, Nanjing University of Posts and Telecommunication, Nanjing, China
Search for more papers by this authorCorresponding Author
Lizhen Deng
National Engineering Research Center of Communication and Network Technology, Nanjing University of Posts and Telecommunication, Nanjing, China
Correspondence
Lizhen Deng, National Engineering Research Center of Communication and Network Technology, Nanjing University of Posts and Telecommunication, Nanjing, China.
Email: [email protected]
Search for more papers by this authorFunding information: National Natural Science Foundation of China, 61701259, 62072256 and NUPTSF : NY220003.
Abstract
Smoke has a very bad effect on the outdoor vision system. Not only are the videos with poor visual effects obtained, but also the quality and structure of the videos are reduced. In this paper, we propose a video smoke removal method based on low-rank tensor completion via spatial-temporal continuity constraint. The proposed method is based on the smoke mixing model and consider the sparseness of smoke and the global and local consistency of clean video. Then, the optimal solution of the smoke removal algorithm model is quickly realized by the Alternating Direction Method of Multiplier. Finally, we evaluate the experiment results of real-world data and simulated data from the visual effects and objective indicators. And the experiment results show that our proposed algorithm can achieve better smoke removal results.
REFERENCES
- 1Dong G, Wang N, Kuang G. Sparse representation of monogenic signal: with application to target recognition in SAR images. IEEE Signal Process Lett. 2014; 21(8): 952-956.
- 2Comaniciu D, Ramesh V, Meer P. Kernel-based object tracking. IEEE Trans Pattern Anal Mach Intell. 2003; 5: 564-575.
- 3Xu G, Khan S, Zhu H, Han L, Ng MK, Yan H. Discriminative tracking via supervised tensor learning. Neurocomputing. 2018; 315: 33-47.
- 4Xu G, Zhu H, Deng L, Han L, Li Y, Lu H. Dilated-aware discriminative correlation filter for visual tracking. WWW. 2019; 22(2): 791-805.
- 5Bolkar S, Wang C, Cheikh FA, Yildirim S. Deep smoke removal from minimally invasive surgery videos. Paper presented at: Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP). Athens, Greece: IEEE; 2018:3403-3407.
- 6Maruta H, Nakamura A, Kurokawa F. A new approach for smoke detection with texture analysis and support vector machine. Paper presented at: Proceedings of the 2010 IEEE International Symposium on Industrial Electronics. Bari, Italy: IEEE; 2010:1550-1555.
- 7Piccinini P, Calderara S, Cucchiara R. Reliable smoke detection in the domains of image energy and color. Paper presented at: Proceedings of the 2008 15th IEEE International Conference on Image Processing. San Diego, California: IEEE; 2008:1376-1379.
- 8Chen TH, Yin YH, Huang SF, Ye YT. The smoke detection for early fire-alarming system base on video processing. Paper presented at: Proceedings of the 2006 International Conference on Intelligent Information Hiding and Multimedia. Pasadena: IEEE; 2006:427-430.
- 9Çelik T, Özkaramanlı H, Demirel H. Fire and smoke detection without sensors: image processing based approach. Paper presented at: Proceedings of the 2007 15th European Signal Processing Conference. Poznań, Poland: IEEE; 2007:1794-1798.
- 10Cheng JYJ. Smoke removal method for ground test image of solid rocket engine based on wavelet transform. J Syst Simulat. 2004; 16(11): 2490-2492.
- 11Serikawa S, Lu H. Underwater image dehazing using joint trilateral filter. Comput Electr Eng. 2014; 40(1): 41-50.
- 12Yu S, Ou W, You X, Mou Y, Jiang X, Tang Y. Single image rain streaks removal based on self-learning and structured sparse representation. Paper presented at: Proceedings of the 2015 IEEE China Summit and International Conference on Signal and Information Processing (ChinaSIP). Chengdu, China: IEEE; 2015:215-219.
- 13Xu M, Jia X, Pickering M, Roberts D. Spectral unmixing for fire smoke detection and removal. Paper presented at: Proceedings of the 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). Bejing, China: IEEE; 2016:806-808.
- 14Nishino K, Kratz L, Lombardi S. Bayesian defogging. Int J Comput Vis. 2012; 98(3): 263-278.
- 15Kotwal A, Bhalodia R, Awate SP. Joint desmoking and denoising of laparoscopy images. Paper presented at: Proceedings of the 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI). Czech Republic Prague, Czech Republic: IEEE; 2016:1050-1054.
- 16Baid A, Kotwal A, Bhalodia R, Merchant S, Awate SP. Joint desmoking, specularity removal, and denoising of laparoscopy images via graphical models and Bayesian inference. Paper presented at: Proceedings of the 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017). Melbourne, Australia: IEEE; 2017:732-736.
- 17Chen WT, Yuan SY, Tsai GC, Wang HC, Kuo SY. Color channel-based smoke removal algorithm using machine learning for static images. Paper presented: 2018 25th IEEE International Conference on Image Processing (ICIP). Athens, Greece: IEEE; 2018:2855-2859.
- 18Zhu H, Qiao Y, Xu G, Deng L, Yu-Feng Y. DSPNet: a lightweight dilated convolution neural networks for spectral deconvolution with self-paced learning. IEEE Trans Ind Inform. 2019; 16(12): 7392–7401.
- 19Lu H, Li Y, Mu S, Wang D, Kim H, Serikawa S. Motor anomaly detection for unmanned aerial vehicles using reinforcement learning. IEEE IoT J. 2017; 5(4): 2315-2322.
- 20Lu H, Li Y, Chen M, Kim H, Serikawa S. Brain intelligence: go beyond artificial intelligence. Mob Netw Appl. 2018; 23(2): 368-375.
- 21Lu H, Wang D, Li Y, et al. CONet: a cognitive ocean network. IEEE Wirel Commun. 2019; 26(3): 90-96.
- 22S L. Research on smoke removal algorithm for grayscale image of fire field. Paper presented at: Proceedings of the National University Safety Science and Engineering Annual Meeting Committee; 2018:1.
- 23Liu F, Ding Y, Rui T, Xu F. Learning low-rank representation with block-sparse structure for single sample face recognition. Proceedings of the International Conference on Internet Multimedia Computing and Service; 2016:238-241; New York, NY, Association for Computing Machinery.
- 24Gao G, Yu Y, Yang M, Chang H, Huang P, Yue D. Cross-resolution face recognition with pose variations via multilayer locality-constrained structural orthogonal procrustes regression. Inf Sci. 2020; 506: 19-36.
- 25Khan S, Nawaz M, Guoxia X, Yan H. Image correspondence with CUR decomposition based graph completion and matching. IEEE Trans Circ Syst Video Technol. 2019; 30(9): 3054–3067.
- 26Yu YF, Xu G, Jiang M, Zhu H, Dai DQ, Yan H. Joint transformation learning via the L2, 1-norm metric for robust graph matching. IEEE Trans Cybern. 2019; 51(2): 521–533.
- 27Li C, Ma Y, Huang J, Mei X, Ma J. Hyperspectral image denoising using the robust low-rank tensor recovery. JOSA A. 2015; 32(9): 1604-1612.
- 28Khan S, Xu G, Chan R, Yan H. An online spatio-temporal tensor learning model for visual tracking and its applications to facial expression recognition. Expert Syst Appl. 2017; 90: 427-438.
- 29Jiang TX, Huang TZ, Zhao XL, Deng LJ, Wang Y. A novel tensor-based video rain streaks removal approach via utilizing discriminatively intrinsic priors. Paper presented at: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI; 2017:4057-4066.
- 30Hore A, Ziou D. Image quality metrics: PSNR vs. SSIM. Paper presented at: Proceedings of the 2010 20th International Conference on Pattern Recognition. Istanbul, Turkey: IEEE; 2010:2366-2369.
- 31Zhang L, Zhang L, Mou X, Zhang D. FSIM: a feature similarity index for image quality assessment. IEEE Trans Image Process. 2011; 20(8): 2378-2386.
- 32He K, Sun J, Tang X. Single image haze removal using dark channel prior. IEEE Trans Pattern Anal Mach Intell. 2010; 33(12): 2341-2353.
- 33Cai B, Xu X, Jia K, Qing C, Tao D. Dehazenet: an end-to-end system for single image haze removal. IEEE Trans Image Process. 2016; 25(11): 5187-5198.
- 34Berman D, Avidan S. Non-local image dehazing. Paper presented at: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV: IEEE; 2016:1674-1682.
- 35Salazar-Colores S, Cruz-Aceves I, Ramos-Arreguin JM. Single image dehazing using a multilayer perceptron. J Electron Imag. 2018; 27(4):043022.
