Pixelwise asphalt concrete pavement crack detection via deep learning-based semantic segmentation method
Ju Huyan
School of Transportation, Southeast University, Nanjing, China
Search for more papers by this authorCorresponding Author
Tao Ma
School of Transportation, Southeast University, Nanjing, China
Correspondence
Tao Ma, School of Transportation, Southeast University, Southeast University Road, Jiangning District, Nanjing 211189, China.
Email: [email protected]
Search for more papers by this authorWei Li
School of Information Engineering, Chang'an University, Xi'an, China
Search for more papers by this authorHanduo Yang
School of Transportation, Southeast University, Nanjing, China
Search for more papers by this authorZhengchao Xu
School of Information Engineering, Chang'an University, Xi'an, China
Search for more papers by this authorJu Huyan
School of Transportation, Southeast University, Nanjing, China
Search for more papers by this authorCorresponding Author
Tao Ma
School of Transportation, Southeast University, Nanjing, China
Correspondence
Tao Ma, School of Transportation, Southeast University, Southeast University Road, Jiangning District, Nanjing 211189, China.
Email: [email protected]
Search for more papers by this authorWei Li
School of Information Engineering, Chang'an University, Xi'an, China
Search for more papers by this authorHanduo Yang
School of Transportation, Southeast University, Nanjing, China
Search for more papers by this authorZhengchao Xu
School of Information Engineering, Chang'an University, Xi'an, China
Search for more papers by this authorFunding information: Fundamental Research Funds for the Central University of China, Grant/Award Numbers: 300102249306, 300102249301; National Key R&D Program of China, Grant/Award Numbers: 2021YFB2600600, 2021YFB2600604; National Natural Science Foundation of China, Grant/Award Number: 52108403
Summary
Explicit gaps exist between the advanced deep learning technologies and the less satisfied pixel-level crack detection algorithms. Therefore, this research sought to bridge this gap via outlining the deep neural network model for pixelwise pavement crack detection. Two state-of-the-art deep neural network models are constructed for the semantic segmentation of crack images. The first architecture, VGGCrackU-net, is composed of 10 convolutional layers, 4 max-pooling layers, 4 up-sampling layers, and 4 concatenate operations. Another architecture, ResCrackU-net, is composed of 7-level residual units with a total of 22 convolutional layers. Asphalt concrete pavement crack images are collected by smartphones, action cameras, and automatic pavement monitoring systems from diverse functional classes of AC pavements. The crack images are manually labeled and double-checked by trained operators for quality insurance. After that, 500 crack images are randomly divided into training, validating, and test datasets according to the ratio of 3:1:1. Both architectures are trained on GPU facilitated Keras platform with Python version of 3.5, which demonstrated fast convergence. Results prove that the proposed models exhibit significant advantages for pixelwise crack detection when compared with the performance of widely used FCN net and PSPnet. Meanwhile, ResCrackU-net slightly outperforms VGGCrackU-net, which, however, can provide acceptable results as well. Though significant false negative and false positive errors are observed in both network models, the contributions are noticeable, which can provide innovative guidance for future work in figuring out solutions to the problems.
Open Research
DATA AVAILABILITY STATEMENT
The dataset used in this research is open available to researchers and can be found in the author's GitHub address (https://github.com/juhuyan/CrackDataset_DL_HY).
REFERENCES
- 1Zhang H, Shen K, Xu G, et al. Fatigue resistance of aged asphalt binders: An investigation of different analytical methods in linear amplitude sweep test. Construct Build Mater. 2020; 241:118099. doi:10.1016/j.conbuildmat.2020.118099
- 2Hu J, Ma T, Zhu Y, Huang X, Xu J. A feasibility study exploring limestone in porous asphalt concrete: Performance evaluation and superpave compaction characteristics. Construct Build Mater. 2021; 279:122457. doi:10.1016/j.conbuildmat.2021.122457
- 3Liu J, Yu B, Hong Q. Molecular dynamics simulation of distribution and adhesion of asphalt components on steel slag. Construct Build Mater. 2020; 255:119332. doi:10.1016/j.conbuildmat.2020.119332
- 4Zhu J, Ma T, Fang Z. Characterization of agglomeration of reclaimed asphalt pavement for cold recycling. Construct Build Mater. 2020; 240:117912. doi:10.1016/j.conbuildmat.2019.117912
- 5Xu G, Fan J, Ma T, Zhao W, Ding X, Wang Z. Research on application feasibility of limestone in sublayer of doublelayer permeable asphalt pavement. Construct Build Mater. 2021; 287:123051. doi:10.1016/j.conbuildmat.2021.123051
- 6Shi C, Cai X, Yi X, Wang T, Yang J. Fatigue crack density of asphalt binders under controlled-stress rotational shear load testing. Construct Build Mater. 2021; 272:121899. doi:10.1016/j.conbuildmat.2020.121899
- 7Wang G, Chen X, Dong Q, Yuan J, Hong Q. Mechanical performance study of pervious concrete using steel slag aggregate through laboratory tests and numerical simulation. J Clean Prod. 2020; 262:121208. doi:10.1016/j.jclepro.2020.121208
- 8Dong Q, Wang G, Chen X, Tan J, Gu X. Recycling of steel slag aggregate in portland cement concrete: An overview. J Clean Prod. 2021; 282:124447. doi:10.1016/j.jclepro.2020.124447
- 9Cui B, Gu X, Hu D, Dong Q. A multiphysics evaluation of the rejuvenator effects on aged asphalt using molecular dynamics simulations. J Clean Prod. 2020; 259:120629. doi:10.1016/j.jclepro.2020.120629
- 10Tang F, Ma T, Zhang J, Guan Y, Chen L. Integrating three-dimensional road design and pavement structure analysis based on BIM. Autom Construct. 2020; 113:103152. doi:10.1016/j.autcon.2020.103152
- 11Ginsberg MD, Shahin MY. Algorithm for crack detection in automated pavement analysis. Proc Infrastruct Plann Manag. 1993; 6: 56-60.
- 12Kaseko M, Ritchie SG, Lo Z-P. Evaluation of two automated thresholding techniques for pavement images. Proc Infrastruct Plann Manag. 1993; 6: 277-281.
- 13Klassen G, Swindall B. Automated crack detection system implementation in ARAN; 1993.
- 14Kaseko MS, Lo ZP, Ritchie SG. Comparison of traditional and neural classifiers for pavement-crack detection. J Transport Eng. 1994; 120(4): 552-569. doi:10.1061/(ASCE)0733-947X
- 15McQueen JM, Timm DH. Statistical analysis of automated versus manual pavement condition surveys. Transp Res Rec. 2005; 1940: 55-62. doi:10.1177/0361198105194000107
10.1177/0361198105194000107 Google Scholar
- 16Li W, Huyan J, Tighe SL. Pavement cracking detection based on three-dimensional data using improved active contour model. J Transp Eng B: Pavements. 2018; 144(2): 04018006. doi:10.1061/JPEODX.0000028
- 17Wang H, Buttlar WG. Three-dimensional micromechanical pavement model development for the study of block cracking. Construct Build Mater. 2019; 206: 35-45. doi:10.1016/j.conbuildmat.2019.01.137
- 18Moussa G, Hussain K. A new technique for automatic detection and parameters estimation of pavement crack. Proceedings IMETI 2011-4th International Multi-Conference on Engineering and Technological Innovation. 2011;2: 11-16.
- 19Adarkwa OA, Attoh-Okine N. Pavement crack classification based on tensor factorization. Construct Build Mater. 2013; 48: 853-857. doi:10.1016/j.conbuildmat.2013.07.091
- 20Delagnes P, Barba D. Markov random field for rectilinear structure extraction in pavement distress image analysis. IEEE Intl Conf Image Process. 1996; 1: 446-449.
10.1109/ICIP.1995.529742 Google Scholar
- 21Lu ZW. Pavement Crack Detection Algorithm Based on Sub-Region and Multi-Scale Analysis. Dongbei Daxue Xuebao/Journal of Northeastern University. 2014; 35(5): 622-625.
- 22Iyer S, Sinha SK. A robust approach for automatic detection and segmentation of cracks in underground pipeline images. Image Vis Comput. 2005; 23(10): 921-933. doi:10.1016/j.imavis.2005.05.017
- 23Liu Y, Li YB, Xie H. Edge detection based on 2D Rosin threshold method in road crack images. Zhongguo Gonglu Xuebao/China Journal of Highway and Transport. 2013; 26(3): 70-76.
- 24Zhang D, Li Q, Chen Y, Cao M, He L, Zhang B. An efficient and reliable coarse-to-fine approach for asphalt pavement crack detection. Image Vis Comput. 2017; 57: 130-146. doi:10.1016/j.imavis.2016.11.018
- 25Wang KCP, Li Q, Gong W. Wavelet-based pavement distress image edge detection with a trous algorithm. Transp Res Rec. 2007; 2024(1): 73-81. doi:10.3141/2024-09
- 26Chambon S, Subirats P, Dumoulin J. Introduction of a wavelet transform based on 2D matched filter in a Markov random field for fine structure extraction: application on road crack detection. Proc SPIE Int Soc Opt Eng. 2009; 7251:72510A.
- 27Mathavan S, Vaheesan K, Kumar A, et al. Detection of pavement cracks using tiled fuzzy Hough transform. J Electron Imag. 2017; 26(5):053008. doi:10.1117/1.JEI.26.5.053008
- 28Hizukuri A, Nagata T. Development of a classification method for a crack on a pavement surface images using machine learning. Proc SPIE Int Soc Opt Eng. 2017; 10338:103380F.
- 29Hoang ND, Nguyen QL. A novel method for asphalt pavement crack classification based on image processing and machine learning. Eng Comput. 2018; 35(2): 1-12. doi:10.1007/s00366-018-0611-9
- 30Hoang ND, Nguyen QL. Automatic recognition of asphalt pavement cracks based on image processing and machine learning approaches: a comparative study on classifier performance. Math Probl Eng. 2018; 2018(1): 16 doi:10.1155/2018/6290498
- 31Tong Z, Gao J, Zhang H. Recognition, location, measurement, and 3D reconstruction of concealed cracks using convolutional neural networks. Construct Build Mater. 2017; 146: 775-787. doi:10.1016/j.conbuildmat.2017.04.097
- 32Kaseko MS, Ritchie SG. A neural network-based methodology for pavement crack detection and classification. Transport Res Part C. 1993; 1(4): 275-291. doi:10.1016/0968-090X(93)90002-W
10.1016/0968-090X(93)90002-W Google Scholar
- 33Hoang ND, Nguyen QL, Bui D. Image processing-based classification of asphalt pavement cracks using support vector machine optimized by artificial bee colony. J Comput Civil Eng. 2018; 32(5):04018037. doi:10.1061/(ASCE)CP.1943-5487.0000781
- 34Zhang D. Crack detection for bituminous pavements based on cluster and minimum spanning tree. Zhongshan Daxue Xuebao/Acta Scientiarum Natralium Universitatis Sunyatseni. 2017; 56(4): 68-74.
- 35Hoang ND, Nguyen QL. Fast local laplacian-based steerable and sobel filters integrated with adaptive boosting classification tree for automatic recognition of asphalt pavement cracks. Adv Civil Eng. 2018; 2018(1): 17 doi:10.1155/2018/5989246
- 36Fujita Y. A method based on machine learning using hand-crafted features for crack detection from asphalt pavement surface images. Proc SPIE Int Soc Opt Eng. 2017; 10338:103380B.
- 37Ji X, Chen Y, Hou Y, Zhen Y. Detecting concealed damage in asphalt pavement based on a composite lead zirconate titanate/polyvinylidene fluoride aggregate. Struct Control Health Monit. 2019;(11): 26, e2452. doi:10.1002/stc.2452
- 38Ersoz AB, Pekcan O, Teke T. Crack identification for rigid pavements using unmanned aerial vehicles. IOP Conf Ser: Mater Sci Eng. 2017; 236(1):012101. doi:10.1088/1757-899X/236/1/012101
10.1088/1757-899X/236/1/012101 Google Scholar
- 39Chen T, Ma T, Huang X, Ma S, Tang F, Wu S. Microstructure of synthetic composite interfaces and verification of mixing order in cold-recycled asphalt emulsion mixture. J Clean Prod. 2020; 263(1):121467. doi:10.1016/j.jclepro.2020.121467
- 40Zou Q, Zhang Z, Li Q, Qi X, Wang Q, Wang S. DeepCrack: learning hierarchical convolutional features for crack detection. IEEE Trans Image Process. 2019; 28(3): 1498-1512. doi:10.1109/TIP.2018.2878966
- 41Huyan J, Li W, Tighe S, Xu Z, Zhai J. CrackU-net: a novel deep convolutional neural network for pixelwise pavement crack detection. Struct Control Health Monit. 2020; 27(8):e2551. doi:10.1002/stc.2551
- 42Wang KCP, Zhang A, Li JQ, Fei Y, Chen C, Li B. Deep learning for asphalt pavement cracking recognition using convolutional neural network. Airfield and Highway Pavements 2017: Design, Construction, Evaluation, and Management of Pavements - Proceedings of the International Conference on Highway Pavements and Airfield Technology 2017. 2017;2017:166-177.
- 43Li Q, Zou Q, Zhang D, Mao Q. FoSA: F* seed-growing approach for crack-line detection from pavement images. Image Vis Comput. 2011; 29(12): 861-872. doi:10.1016/j.imavis.2011.10.003
- 44Liu J, Yu B, Wang Q. Application of steel slag in cement treated aggregate base course. J Clean Prod. 2020; 269(1):121733. doi:10.1016/j.jclepro.2020.121733
- 45Zhu J, Ma T, Dong Z. Evaluation of optimum mixing conditions for rubberized asphalt mixture containing reclaimed asphalt pavement. Construct Build Mater. 2020; 234:117426. doi:10.1016/j.conbuildmat.2019.117426
- 46Huang M, Dong Q, Ni F, Wang L. LCA and LCCA based multi-objective optimization of pavement maintenance. J Clean Prod. 2021; 283(10): 124583. doi:10.1016/j.jclepro.2020.124583
- 47Gao S., Jie Z., Pan Z., Qin F., Li R., Automatic recognition of pavement crack via convolutional neural network, in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 2018. p. 82–89. doi:10.1007/978-3-662-56689-3_7
10.1007/978-3-662-56689-3_7 Google Scholar
- 48Kim B, Cho S. Image-based concrete crack assessment using mask and region-based convolutional neural network. Struct Control Health Monit. 2019; 26(8):e2381. doi:10.1002/stc.2381e2381
- 49Wang X, Hu Z. Grid-based pavement crack analysis using deep learning. 2017 4th International Conference on Transportation Information and Safety, ICTIS 2017 - Proceedings. 2017;20:917-924.
- 50Yang J, Li Z, Xu X. Preparation and evaluation of cooling asphalt concrete modified with SBS and tourmaline anion powder. J Clean Prod. 2021; 289(20):125135. doi:10.1016/j.jclepro.2020.125135
- 51Hu J, Ma T, Ma K. DEM-CFD simulation on clogging and degradation of air voids in double-layer porous asphalt pavement under rainfall. J Hydrol. 2021; 595:126028. doi:10.1016/j.jhydrol.2021.126028
- 52Zhang A, Wang KCP, Li B, et al. Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network. Comput Aided Civ Inf Eng. 2017; 32(10): 805-819. doi:10.1111/mice.12297
- 53Zhang A, Wang KCP, Fei Y, et al. Deep learning-based fully automated pavement crack detection on 3D asphalt surfaces with an improved CrackNet. J Comput Civil Eng. 2018; 32(5):04018041. doi:10.1061/(ASCE)CP.1943-5487.000077504018041
- 54Zhang A, Wang KCP, Fei Y, et al. Automated pixel-level pavement crack detection on 3D asphalt surfaces with a recurrent neural network. Comput Aided Civ Inf Eng. 2019; 34(3): 213-229. doi:10.1111/mice.12409
- 55Bang S, Park S, Kim H, Kim H. Encoder–decoder network for pixel-level road crack detection in black-box images. Comput Aided Civ Inf Eng. 2019; 34(8): 713-727. doi:10.1111/mice.12440
- 56Ni F, Zhang J, Chen ZQ. Pixel-level crack delineation in images with convolutional feature fusion. 2019; 26(1):e2286. doi:10.1002/stc.2286
- 57Tang Y, Zhang AA, Luo L, Wang G, Yang E. Pixel-level pavement crack segmentation with encoder-decoder network. Measurement. 2021; 184(1):109914. doi:10.1016/j.measurement.2021.109914
- 58Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift; 2015.
- 59Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition; 2014.
- 60He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016;2016: 770-778.
- 61Gulli A, Pal S. Deep Learning with Keras. Packt Publishing Ltd; 2017.