Automatic pixel-level crack detection and evaluation of concrete structures using deep learning
Weijian Zhao
College of Civil Engineering and Architecture, Zhejiang University, Hangzhou, China
Contribution: Conceptualization, Methodology, Validation, Writing - original draft, Writing - review & editing, Supervision
Search for more papers by this authorYunyi Liu
College of Civil Engineering and Architecture, Zhejiang University, Hangzhou, China
Contribution: Methodology, Validation, Writing - original draft, Writing - review & editing
Search for more papers by this authorJiawei Zhang
College of Civil Engineering and Architecture, Zhejiang University, Hangzhou, China
Contribution: Conceptualization, Methodology
Search for more papers by this authorYi Shao
Dept. of Civil and Environmental Engineering, Stanford University, Stanford, California, USA
Contribution: Conceptualization
Search for more papers by this authorCorresponding Author
Jiangpeng Shu
College of Civil Engineering and Architecture, Zhejiang University, Hangzhou, China
Correspondence
Jiangpeng Shu, College of Civil Engineering and Architecture, Zhejiang University, 310058 Hangzhou, China.
Email: [email protected]
Contribution: Conceptualization, Supervision
Search for more papers by this authorWeijian Zhao
College of Civil Engineering and Architecture, Zhejiang University, Hangzhou, China
Contribution: Conceptualization, Methodology, Validation, Writing - original draft, Writing - review & editing, Supervision
Search for more papers by this authorYunyi Liu
College of Civil Engineering and Architecture, Zhejiang University, Hangzhou, China
Contribution: Methodology, Validation, Writing - original draft, Writing - review & editing
Search for more papers by this authorJiawei Zhang
College of Civil Engineering and Architecture, Zhejiang University, Hangzhou, China
Contribution: Conceptualization, Methodology
Search for more papers by this authorYi Shao
Dept. of Civil and Environmental Engineering, Stanford University, Stanford, California, USA
Contribution: Conceptualization
Search for more papers by this authorCorresponding Author
Jiangpeng Shu
College of Civil Engineering and Architecture, Zhejiang University, Hangzhou, China
Correspondence
Jiangpeng Shu, College of Civil Engineering and Architecture, Zhejiang University, 310058 Hangzhou, China.
Email: [email protected]
Contribution: Conceptualization, Supervision
Search for more papers by this authorFunding information: National Natural Science Foundation of China, Grant/Award Number: 52108179; Key R&D Program of Zhejiang Province, Grant/Award Number: 2021C01106
Summary
To achieve effective inspection of reinforced concrete structures, a smart solution to obtain quantitative crack information automatically from inspection is essential. Therefore, a novel crack feature pyramid network (Crack-FPN) based on image analysis is proposed, which has a distinctive feature extraction ability and reduced computational cost. In this method, the network is trained using public annotated pixel-level image datasets. Additionally, beam loading tests and image dataset collection were performed to validate the effectiveness of the proposed method. The crack regions of test images are detected by YOLOv5 using bounding boxes and segmented by Crack-FPN. Compared to existing methods, Crack-FPN shows higher detection accuracy and computational efficiency for crack images affected by illumination conditions and complex backgrounds. In terms of crack width estimation, the relative error of the proposed method in a wide crack width range is approximately 5%, which is considerably small compared with manual measurement results.
Open Research
DATA AVAILABILITY STATEMENT
The data that support the findings of this study are available from the corresponding author upon reasonable request.
REFERENCES
- 1Ye XW, Jin T, Yun CB. A review on deep learning-based structural health monitoring of civil infrastructures. Smart Struct Syst. 2019; 24(5): 567-585. doi:10.12989/sss.2019.24.5.567
- 2Shu J, Plos M, Zandi K, Ashraf. Distribution of shear force: a multi-level assessment of a cantilever RC slab. Eng Struct. 2019; 190(February): 345-359. doi:10.1016/j.engstruct.2019.04.045
- 3Shu J, Honfi D, Plos M, Zandi K, Magnusson. Assessment of a cantilever bridge deck slab using multi-level assessment strategy and decision support framework. Eng Struct. 2019; 200:109666. doi:10.1016/j.engstruct.2019.109666
- 4Ng HF. Automatic thresholding for defect detection. Pattern Recogn Lett. 2006; 27(14): 1644-1649. doi:10.1016/j.patrec.2006.03.009
- 5Beck A, Teboulle M. Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems. IEEE Trans Image Process. 2009; 18(11): 2419-2434. doi:10.1109/TIP.2009.2028250
- 6Hutchinson TC, Chen Z. Improved image analysis for evaluating concrete damage. J Comput Civ Eng. 2006; 20(3): 210-216. doi:10.1061/(asce)0887-3801(2006)20:3(210)
- 7Qu Z, Lin LD, Guo Y, Wang N. An improved algorithm for image crack detection based on percolation model. IEEJ Trans Electr Electron Eng. 2015; 10(2): 214-221. doi:10.1002/tee.22056
- 8LeCun Y, Boser B, Denker JS, et al. Backpropagation applied to digit recognition. Neural Comput. 1989; 1(4): 541-551. doi:10.1162/neco.1989.1.4.541
- 9Shelhamer E, Long J, Darrell T. Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell. 2017; 39(4): 640-651. doi:10.1109/TPAMI.2016.2572683
- 10Ronneberger O, Fischer P, Brox. U-net: convolutional networks for biomedical image segmentation. Lect Notes Comput Sci (Including Subser Lect Notes Artif Intell Lect Notes Bioinformatics). 2015; 9351: 234-241. doi:10.1007/978-3-319-24574-4_28
10.1007/978‐3‐319‐24574‐4_28 Google Scholar
- 11Yu F, Koltun V. Multi-scale context aggregation by dilated convolutions. 4th Int Conf Learn Represent ICLR 2016 - Conf Track Proc. Published online. 2016.
- 12Lin G, Milan A, Shen C, Reid I. RefineNet: multi-path refinement networks for high-resolution semantic segmentation. Proc - 30th IEEE Conf Comput Vis Pattern Recognition, CVPR 2017. 2017; 5168-5177. doi:10.1109/CVPR.2017.549
- 13Lin T-Y, Dollar P, Girshick R, He K, Hariharan B, Belongie S. Feature pyramid networks for object detection. Proc IEEE Conf Comput Vis Pattern Recognit. 2017; 936-944. doi:10.1109/ISPA-BDCloud-SustainCom-SocialCom48970.2019.00217
- 14Cha YJ, Choi W, Suh G, Mahmoudkhani S, Büyüköztürk O. Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Comput Civ Infrastruct Eng. 2018; 33(9): 731-747. doi:10.1111/mice.12334
- 15Ghosh Mondal T, Jahanshahi MR, Wu RT, Wu ZY. Deep learning-based multi-class damage detection for autonomous post-disaster reconnaissance. Struct Control Health Monit. 2020; 27(4): 1-15. doi:10.1002/stc.2507
- 16Mohtasham Khani M, Vahidnia S, Ghasemzadeh L, et al. Deep-learning-based crack detection with applications for the structural health monitoring of gas turbines. Struct Heal Monit. 2020; 19(5): 1440-1452. doi:10.1177/1475921719883202
- 17Atha DJ, Jahanshahi MR. Evaluation of deep learning approaches based on convolutional neural networks for corrosion detection. Struct Heal Monit. 2018; 17(5): 1110-1128. doi:10.1177/1475921717737051
- 18Du Y, Pan N, Xu Z, Deng F, Shen Y, Kang H. Pavement distress detection and classification based on YOLO network. Int J Pavement Eng. 2020; 0(0): 1-14. doi:10.1080/10298436.2020.1714047
- 19Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: unified, real-time object detection. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit. 2016;2016: 779-788. doi:10.1109/CVPR.2016.91
- 20Chen FC, Jahanshahi MR. NB-CNN: deep learning-based crack detection using convolutional neural network and naïve Bayes data fusion. IEEE Trans Ind Electron. 2018; 65(5): 4392-4400. doi:10.1109/TIE.2017.2764844
- 21Xue Y, Li Y. A fast detection method via region-based fully convolutional neural networks for shield tunnel lining defects. Comput Civ Infrastruct Eng. 2018; 33(8): 638-654. doi:10.1111/mice.12367
- 22Zhang X, Rajan D, Story B. Concrete crack detection using context-aware deep semantic segmentation network. Comput Civ Infrastruct Eng. 2019; 34(11): 951-971. doi:10.1111/mice.12477
- 23Cha YJ, Choi W, Büyüköztürk O. Deep learning-based crack damage detection using convolutional neural networks. Comput Civ Infrastruct Eng. 2017; 32(5): 361-378. doi:10.1111/mice.12263
- 24Ye XW, Jin T, Chen PY. Structural crack detection using deep learning–based fully convolutional networks. Adv Struct Eng. 2019; 22(16): 3412-3419. doi:10.1177/1369433219836292
- 25Liu Z, Cao Y, Wang Y, Wang. Computer vision-based concrete crack detection using U-net fully convolutional networks. Autom Constr. 2019; 104: 129-139. doi:10.1016/j.autcon.2019.04.005
- 26Huyan J, Li W, Tighe S, Xu Z, Zhai J. CrackU-net: a novel deep convolutional neural network for pixelwise pavement crack detection. Struct Control Health Monit. 2020; 27(8): 1-19. doi:10.1002/stc.2551
- 27Ni FT, Zhang J, Chen ZQ. Pixel-level crack delineation in images with convolutional feature fusion. Struct Control Health Monit. 2019; 26(1): 1-18. doi:10.1002/stc.2286
- 28Liu Y, Yao J, Lu X, Xie R, Li. DeepCrack: a deep hierarchical feature learning architecture for crack segmentation. Neurocomputing. 2019; 338: 139-153. doi:10.1016/j.neucom.2019.01.036
- 29Jiang S, Zhang J. Real-time crack assessment using deep neural networks with wall-climbing unmanned aerial system. Comput Civ Infrastruct Eng. 2020; 35(6): 549-564. doi:10.1111/mice.12519
- 30Tong Z, Gao J, Han Z, Wang Z. Recognition of asphalt pavement crack length using deep convolutional neural networks. Road Mater Pavement des. 2018; 19(6): 1334-1349. doi:10.1080/14680629.2017.1308265
- 31Ni FT, Zhang J, Chen ZQ. Zernike-moment measurement of thin-crack width in images enabled by dual-scale deep learning. Comput Civ Infrastruct Eng. 2019; 34(5): 367-384. doi:10.1111/mice.12421
- 32Alipour M, Harris DK, Miller GR. Robust pixel-level crack detection using deep fully convolutional neural networks. J Comput Civ Eng. 2019; 33(6): 1-14. doi:10.1061/(ASCE)CP.1943-5487.0000854
- 33Kang D, Benipal SS, Gopal DL, Cha YJ. Hybrid pixel-level concrete crack segmentation and quantification across complex backgrounds using deep learning. Autom Constr. 2020; 118:103291. doi:10.1016/j.autcon.2020.103291
- 34Park SE, Eem SH, Jeon. Concrete crack detection and quantification using deep learning and structured light. Constr Build Mater. 2020; 252:119096. doi:10.1016/j.conbuildmat.2020.119096
- 35Koch C, Georgieva K, Kasireddy V, Akinci B, Fieguth P. A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure. Adv Eng Informatics. 2015; 29(2): 196-210. doi:10.1016/j.aei.2015.01.008
- 36Yang Y, Nagarajaiah. Dynamic imaging: real-time detection of local structural damage with blind separation of low-rank background and sparse innovation. J Struct Eng. 2016; 142(2):04015144. doi:10.1061/(asce)st.1943-541x.0001334
- 37Bhowmick S, Nagarajaiah S. Automatic detection and damage quantification of multiple cracks on concrete surface from video. Int J Sustain Mater Struct Syst. 2020; 4.2-4(2/3/4): 292-311. doi:10.1504/IJSMSS.2020.109097
10.1504/IJSMSS.2020.109097 Google Scholar
- 38Hu J, Shen L, Albanie S, Sun G, Wu E. Squeeze-and-Excitation Networks. IEEE Trans Pattern Anal Mach Intell. 2020; 42(8): 2011-2023. doi:10.1109/TPAMI.2019.2913372
- 39Redmon J, Farhadi A. YOLO9000: Better, faster, stronger. Proc - 30th IEEE Conf Comput Vis Pattern Recognition, CVPR 2017. 2017; 6517-6525. doi:10.1109/CVPR.2017.690
- 40Redmon J, Farhadi A. YOLO v.3. Tech Rep. 2018; 1-6. https://pjreddie.com/media/files/papers/YOLOv3.pdf
- 41Bochkovskiy A, Wang C-Y, Liao H-YM. YOLOv4: optimal speed and accuracy of object detection. 2020. http://arxiv.org/abs/2004.10934
- 42Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. 3rd Int Conf Learn Represent ICLR 2015 - Conf Track Proc. 2015; 1-14.
- 43He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit. 2016. 2016; 770-778. doi:10.1109/CVPR.2016.90
- 44Xie S, Girshick R, Dollár P, Tu Z, He K. Aggregated residual transformations for deep neural networks. Proc - 30th IEEE Conf Comput Vis Pattern Recognition, CVPR 2017. 2017; 5987-5995. doi:10.1109/CVPR.2017.634
- 45Richard P, Cheyrezy M. Composition of Reactive Powder Concretes. Cem Concr Res. 1995; 25(7): 1501-1511. doi:10.1016/0008-8846(95)00144-2
- 46Graybeal B, Brühwiler E, Kim B-S, Toutlemonde F, Voo YL, Zaghi A. International Perspective on UHPC in Bridge Engineering. J Bridg Eng. 2020; 25(11):04020094. doi:10.1061/(asce)be.1943-5592.0001630
- 47Shao Y. Improving ductility and design methods of reinforced high-performance fiber-reinforced cementitious composite (HPFRCC) flexural members. 2020.
- 48Mundt M, Majumder S, Murali S, Panetsos P, Ramesh V. Meta-learning convolutional neural architectures for multi-target concrete defect classification with the concrete defect bridge image dataset. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit. 2019. 2019; 11188-11197. doi:10.1109/CVPR.2019.01145
- 49Garcia-Garcia A, Orts-Escolano S, Oprea S, Villena-Martinez V, Garcia-Rodriguez J. A review on deep learning techniques applied to semantic segmentation. 2017: 1-23. doi:10.1016/j.asoc.2018.05.018
- 50Milletari F, Navab N, Ahmadi SA. V-Net: fully convolutional neural networks for volumetric medical image segmentation. Proc - 2016 4th Int Conf 3D Vision, 3DV 2016. 2016; 565-571. doi:10.1109/3DV.2016.79
- 51Kingma DP, Ba JL. Adam: A method for stochastic optimization. 3rd Int Conf Learn Represent ICLR 2015 - Conf Track Proc. 2015; 1-15.
- 52Li S, Zhao X, Zhou G. Automatic pixel-level multiple damage detection of concrete structure using fully convolutional network. Comput Civ Infrastruct Eng. 2019; 34(7): 616-634. doi:10.1111/mice.12433
- 53Gehri N, Mata-Falcón J, Kaufmann W. Automated crack detection and measurement based on digital image correlation. Constr Build Mater. 2020; 256:119383. doi:10.1016/j.conbuildmat.2020.119383
- 54Wang W, Zhang A, Wang KCP, Braham AF, Qiu S. Pavement crack width measurement based on Laplace's equation for continuity and unambiguity. Comput Civ Infrastruct Eng. 2018; 33(2): 110-123. doi:10.1111/mice.12319
- 55Bhowmick S, Nagarajaiah S, Veeraraghavan. Vision and deep learning-based algorithms to detect and quantify cracks on concrete surfaces from UAV videos. Sensors (Switzerland). 2020; 20(21): 1-19. doi:10.3390/s20216299
