Volume 23 Issue 2
Apr.  2023
Turn off MathJax
Article Contents
Article Navigation >  Journal of Traffic and Transportation Engineering  > 2023  >  23(2): 166-182
ZHANG Wei-guang, ZHONG Jing-tao, HUYAN Ju, MA Tao, ZHU Jun-qing, HE Liang. Extraction and quantification of pavement alligator crack morphology based on VGG16-UNet semantic segmentation model[J]. Journal of Traffic and Transportation Engineering, 2023, 23(2): 166-182. doi: 10.19818/j.cnki.1671-1637.2023.02.012
Citation: ZHANG Wei-guang, ZHONG Jing-tao, HUYAN Ju, MA Tao, ZHU Jun-qing, HE Liang. Extraction and quantification of pavement alligator crack morphology based on VGG16-UNet semantic segmentation model[J]. Journal of Traffic and Transportation Engineering, 2023, 23(2): 166-182. doi: 10.19818/j.cnki.1671-1637.2023.02.012
shu

Extraction and quantification of pavement alligator crack morphology based on VGG16-UNet semantic segmentation model

doi: 10.19818/j.cnki.1671-1637.2023.02.012
  • 1.

    School of Transportation, Southeast University, Nanjing 210096, Jiangsu, China

  • 2.

    School of Civil Engineering, Chongqing Jiaotong University, Chongqing 400074, China

Funds:

National Key Research and Development Program of China 2020YFB1600102

National Natural Science Foundation of China 52278443

More Information
  • Author Bio:

    ZHANG Wei-guang(1986-), male, associate professor, PhD, wgzhang@seu.edu.cn

    MA Tao(1981-), male, professor, PhD, matao@seu.edu.cn

  • Received Date: 2022-11-04
    Available Online: 2023-05-09
  • Publish Date: 2023-04-25
    Abstract
  • A pavement alligator crack segmentation technology was proposed based on the VGG16-UNet semantic segmentation model, and a quantitative evaluation method was developed for extracting alligator crack morphology via the proposed model. Alligator crack images were collected by a mobile camera under the conditions of dry pavement, waterlogged pavement, and road marking. A multi-scene alligator crack image dataset was established including light, medium, and heavy damages. The alligator crack image segmentation effects and corresponding segmentation indexes of VGG16-UNet, VGG19-UNet, PSPNet, SegNet, and DeepLab v3 + were analyzed to select the optimal segmentation model, and the alligator crack morphology features were extracted. Three contour fitting boundary methods of the circumscribed regular rectangle, minimum circumscribed rectangle, and convex hull were compared. The optimal fitting boundary was determined, and the area of alligator cracks was calculated. A calculation method of alligator crack fragmentation based on the coordinates of the extreme points of the contour boundary was proposed. The pavement alligator crack area and fragmentation features were analyzed accordingly. Research results show that compared with four semantic segmentation models of VGG19-UNet, PSPNet, SegNet, and DeepLab v3 +, VGG16-UNet has the smallest model parameter quantity of 14 710 464, and the training speed can reach 118 s for each training epoch. It possesses the highest prediction speed of 0.021 s for each image. Additionally, VGG16-UNet shows a precision of 81%, recall of 82%, harmonic mean of 0.81, and mean intersection over union (MIoU) of 0.73 on open-source Crack500 dataset, CrackTree dataset, and EdmCrack600 dataset. It outperforms the other four models and shows higher generalization ability. The proposed convex hull boundary fitting method gets the smallest alligator crack area. Compared with the boundary fitting methods of circumscribed regular rectangle and minimum circumscribed rectangle, the proposed method increases the fitting rate by 14.47% and 9.30%, respectively, and obtains the optimal pavement alligator crack contour boundary. The alligator crack fragmentation values calculated by the extreme points of the contour boundary can evaluate the damage level of alligator cracks, which solves the difficulty of existing pavement alligator crack fragmentation calculation.

     

    • FullText(HTML)
  • loading
    • References(50)
  • [1]
    MAEDA H, SEKIMOTO Y, SETO T, et al. Road damage detection and classification using deep neural networks with smartphone images[J]. Computer-Aided Civil and Infrastructure Engineering, 2018, 33(12): 1127-1141. doi: 10.1111/mice.12387
    [2]
    LI B X, WANG K C P, ZHANG A, et al. Automatic classification of pavement crack using deep convolutional neural network[J]. International Journal of Pavement Engineering, 2020, 21(4): 457-463. doi: 10.1080/10298436.2018.1485917
    [3]
    ZHANG A, WANG K C P, LI B X, et al. Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network[J]. Computer-Aided Civil and Infrastructure Engineering, 2017, 32(10): 805-819. doi: 10.1111/mice.12297
    [4]
    TANG Fan-long, MA Tao, ZHANG Jun-hui, et al. Integrating three-dimensional road design and pavement structure analysis based on BIM[J]. Automation in Construction, 2020, 113: 103152. doi: 10.1016/j.autcon.2020.103152
    [5]
    TANG Fan-long, MA Tao, GUAN Yong-sheng, et al. Parametric modeling and structure verification of asphalt pavement based on BIM-ABAQUS[J]. Automation in Construction, 2020, 111: 103066. doi: 10.1016/j.autcon.2019.103066
    [6]
    曹佳煜. 基于图像处理的路面裂缝自动检测技术研究[D]. 西安: 长安大学, 2014.

    CAO Jia-yu. Research on the technology of automatic pavement crack detection based on digital image processing[D]. Xi'an: Chang'an University, 2014. (in Chinese)
    [7]
    PACHECO-CHERREZ J, CARDENAS D, PROBST O. Measuring crack-type damage features in thin-walled composite beams using denoising and a 2D continuous wavelet transform of mode shapes[J]. Journal of Applied and Computational Mechanics, 2021, 7(1): 355-371.
    [8]
    李东明, 盖梦野, 李超然, 等. 基于小波域的Contourlet变换法的自适应光学图像去噪算法研究[J]. 激光与光电子学进展, 2015, 52(11): 97-102. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ201511013.htm

    LI Dong-ming, GAI Meng-ye, LI Chao-ran, et al. Research on adaptive optics image denoising algorithm based on the wavelet-based contourlet transform[J]. Laser and Optoelectronics Progress, 2015, 52(11): 97-102. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ201511013.htm
    [9]
    王世芳, 车艳丽, 李楠, 等. 一种基于多尺度脊边缘的沥青路面裂缝检测算法[J]. 中国公路学报, 2017, 30(4): 32-41. doi: 10.3969/j.issn.1001-7372.2017.04.005

    WANG Shi-fang, CHE Yan-li, LI Nan, et al. Asphalt pavement crack detection algorithm based on multi scale ridges[J]. China Journal of Highway and Transport, 2017, 30(4): 32-41. (in Chinese) doi: 10.3969/j.issn.1001-7372.2017.04.005
    [10]
    马文涛, 樊春玲. 基于计算机视觉路面裂缝的识别与测量[J]. 电子测量技术, 2020, 43(19): 85-88. https://www.cnki.com.cn/Article/CJFDTOTAL-DZCL202019018.htm

    MA Wen-tao, FAN Chun-ling. Recognition and measurement of pavement cracks based on computer vision[J]. Electronic Measurement Technology, 2020, 43(19): 85-88. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-DZCL202019018.htm
    [11]
    彭博, 蒋阳升, 韩世凡, 等. 路面裂缝图像自动识别算法综述[J]. 公路交通科技, 2014, 31(7): 19-25. doi: 10.3969/j.issn.1002-0268.2014.07.004

    PENG Bo, JIANG Yang-sheng, HAN Shi-fan, et al. A review of automatic pavement crack image recognition algorithms[J]. Journal of Highway and Transportation Research and Development, 2014, 31(7): 19-25. (in Chinese) doi: 10.3969/j.issn.1002-0268.2014.07.004
    [12]
    张红. 基于图像处理的路面裂缝检测系统研究[D]. 沈阳: 沈阳航空航天大学, 2018.

    ZHANG Hong. Research on pavement crack detection system based on image processing[D]. Shenyang: Shenyang Aerospace University, 2018. (in Chinese)
    [13]
    DU Zhen-yu, YUAN Jie, XIAO Fei-peng, et al. Application of image technology on pavement distress detection: a review[J]. Measurement, 2021, 184: 109900. doi: 10.1016/j.measurement.2021.109900
    [14]
    MEI Qi-pei, GÜL M, AZIM M. R. Densely connected deep neural network considering connectivity of pixels for automatic crack detection[J]. Automation in Construction, 2020, 110: 103018. doi: 10.1016/j.autcon.2019.103018
    [15]
    张伟光, 钟靖涛, 于建新, 等. 基于机器学习和图像处理的路面裂缝检测技术研究[J]. 中南大学学报(自然科学版), 2021, 52(7): 2402-2415. https://www.cnki.com.cn/Article/CJFDTOTAL-ZNGD202107028.htm

    ZHANG Wei-guang, ZHONG Jing-tao, YU Jian-xin, et al. Research on pavement crack detection technology based on convolution neural network[J]. Journal of Central South University (Science and Technology), 2021, 52(7): 2402-2415. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZNGD202107028.htm
    [16]
    ZHU Jun-qing, ZHONG Jing-tao, MA Tao, et al. Pavement distress detection using convolutional neural networks with images captured via UAV[J]. Automation in Construction, 2022, 133: 103991. doi: 10.1016/j.autcon.2021.103991
    [17]
    CHA Y J, CHOI W, BÜYÜKÖZTÜRK O. Deep learning- based crack damage detection using convolutional neural networks[J]. Computer-Aided Civil and Infrastructure Engineering, 2017, 32(5): 361-378. doi: 10.1111/mice.12263
    [18]
    DU Yu-chuan, PAN Ning, XU Zi-hao, et al. Pavement distress detection and classification based on YOLO network[J]. International Journal of Pavement Engineering, 2021, 22(13): 1659-1672. doi: 10.1080/10298436.2020.1714047
    [19]
    YANG Xin-cong, LI Heng, YU Yan-tao, et al. Automatic pixel-level crack detection and measurement using fully convolutional network[J]. Computer-Aided Civil and Infrastructure Engineering, 2018, 33(12): 1090-1109. doi: 10.1111/mice.12412
    [20]
    SHELHAMER E, LONG J, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651. doi: 10.1109/TPAMI.2016.2572683
    [21]
    RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//S pringer. Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin: Springer, 2015: 234-241.
    [22]
    BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495. doi: 10.1109/TPAMI.2016.2644615
    [23]
    CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834-848. doi: 10.1109/TPAMI.2017.2699184
    [24]
    ZOU Qin, ZHANG Zheng, LI Qing-quan, et al. DeepCrack: Learning hierarchical convolutional features for crack detection[J]. IEEE Transactions on Image Processing, 2019, 28(3): 1498-1512. doi: 10.1109/TIP.2018.2878966
    [25]
    KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90. doi: 10.1145/3065386
    [26]
    SZEGEDY C, LIU W, JIA Y Q, et al. Going deeper with convolutions[C]//IEEE. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2015: 1-9.
    [27]
    SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]//ICLR. 3rd International Conference on Learning Representations. Washington DC: ICLR, 2015: 1-14.
    [28]
    HE Kai-ming, ZHANG Xiang-yu, REN Shao-qing, et al. Deep residual learning for image recognition[C]//IEEE. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2016: 770-778.
    [29]
    SHI Ji-yuan, DANG Ji, CUI Mi-da, et al. Improvement of damage segmentation based on pixel-level data balance using VGG-UNet[J]. Applied Sciences, 2021, 11: 518. doi: 10.3390/app11020518
    [30]
    LIU Z, SONG Y Q, SHENG V S, et al. Liver CT sequence segmentation based with improved U-Net and graph cut[J]. Expert Systems with Applications, 2019, 126: 54-63. doi: 10.1016/j.eswa.2019.01.055
    [31]
    LIAN Sheng, LUO Zhi-ming, ZHONG Zhun, et al. Attention guided U-Net for accurate iris segmentation[J]. Journal of Visual Communication and Image Representation, 2018, 56: 296-304. doi: 10.1016/j.jvcir.2018.10.001
    [32]
    LI Gang, MA Biao, HE Shuan-hai, et al. Automatic tunnel crack detection based on U-Net and a convolutional neural network with alternately updated clique[J]. Sensors, 2020, 20(3): 717. doi: 10.3390/s20030717
    [33]
    朱苏雅, 杜建超, 李云松, 等. 采用U-Net卷积网络的桥梁裂缝检测方法[J]. 西安电子科技大学学报, 2019, 46(4): 35-42. https://www.cnki.com.cn/Article/CJFDTOTAL-XDKD201904006.htm

    ZHU Su-ya, DU Jian-chao, LI Yun-song, et al. Method for bridge crack detection based on the U-Net convolutional networks[J]. Journal of Xidian University, 2019, 46(4): 35-42. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-XDKD201904006.htm
    [34]
    王海, 蔡柏湘, 蔡英凤, 等. 基于语义分割网络的路面积水与湿滑区域检测[J]. 汽车工程, 2021, 43(4): 485-491. https://www.cnki.com.cn/Article/CJFDTOTAL-QCGC202104007.htm

    WANG Hai, CAI Bai-xiang, CAI Ying-feng, et al. Detection of water-covered and wet areas on road pavement based on semantic segmentation network[J]. Automotive Engineering, 2021, 43(4): 485-491. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-QCGC202104007.htm
    [35]
    LIU Zheng-qing, CAO Yi-wen, WANG Yi-ze, et al. Computer vision-based concrete crack detection using U-net fully convolutional networks[J]. Automation in Construction, 2019, 104: 129-139. doi: 10.1016/j.autcon.2019.04.005
    [36]
    HUYAN Ju, LI Wei, TIGHE S, et al. CrackU-net: a novel deep convolutional neural network for pixelwise pavement crack detection[J]. Structural Control and Health Monitoring, 2020, 27: e2551.
    [37]
    沙爱民, 童峥, 高杰. 基于卷积神经网络的路表病害识别与测量[J]. 中国公路学报, 2018, 31(1): 1-10. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL201801002.htm

    SHA Ai-min, TONG Zheng, GAO Jie. Recognition and measurement of pavement disasters based on convolutional neural networks[J]. China Journal of Highway and Transport, 2018, 31(1): 1-10. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL201801002.htm
    [38]
    李海东. 基于深度学习和无人机的重交通荷载公路病害定位和测量研究[D]. 西安: 长安大学, 2019.

    LI Hai-dong. Research on pavement distresses location and measurement under the heavy traffic using deep learning and UAV[D]. Xi'an: Chang'an University, 2019. (in Chinese)
    [39]
    晏班夫, 徐观亚, 栾健, 等. 基于Faster R-CNN与形态法的路面病害识别[J]. 中国公路学报, 2021, 34(9): 181-193. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL202109016.htm

    YAN Ban-fu, XU Guan-ya, LUAN Jian, et al. Pavement distress detection based on Faster R-CNN and morphological operations[J]. China Journal of Highway and Transport, 2021, 34(9): 181-193. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL202109016.htm
    [40]
    LIU Jing-wei, YANG Xu, LAU S, et al. Automated pavement crack detection and segmentation based on two-step convolutional neural network[J]. Computer-Aided Civil and Infrastructure Engineering, 2020, 35(11): 1291-1305.
    [41]
    孙朝云, 马志丹, 李伟, 等. 基于深度卷积神经网络融合模型的路面裂缝识别方法[J]. 长安大学学报(自然科学版), 2020, 40(4): 1-13. https://www.cnki.com.cn/Article/CJFDTOTAL-XAGL202004002.htm

    SUN Zhao-yun, MA Zhi-dan, LI Wei, et al. Pavement crack identification method based on deep convolutional neural network fusion model[J]. Journal of Chang'an University (Natural Science Edition), 2020, 40(4): 1-13. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-XAGL202004002.htm
    [42]
    王建明, 黎向锋, 叶磊, 等. 基于信道注意结构的生成对抗网络医学图像去模糊[J]. 计算机科学, 2021, 48(S1): 101-106. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJA2021S1018.htm

    WANG Jian-ming, LI Xiang-feng, YE Lei, et al. Medical image deblur using generative adversarial networks with channel attention[J]. Computer Science, 2021, 48(S1): 101-106. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JSJA2021S1018.htm
    [43]
    ZHAO Heng-shuang, SHI Jian-ping, QI Xiao-juan, et al. Pyramid scene parsing network[C]//IEEE. Proceedings of 30th IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2017: 6230-6239.
    [44]
    CHEN L C, ZHU Y C, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Springer. 15th European Conference on Computer Vision (ECCV). Berlin: Springer, 2018: 833-851.
    [45]
    YANG Fan, ZHANG Lei, YU Si-jia, et al. Feature pyramid and hierarchical boosting network for pavement crack detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 21(4): 1525-1535.
    [46]
    ZOU Qin, CAO Yu, LI Qing-zhou, et al. CrackTree: automatic crack detection from pavement images[J]. Pattern Recognition Letters, 2012, 33(3): 227-238.
    [47]
    MEI Qi-pei, GÜL M. A cost effective solution for pavement crack inspection using cameras and deep neural networks[J]. Construction and Building Materials, 2020, 256: 119397.
    [48]
    彭博, 蒋阳升, 蒲云. 基于数字图像处理的路面裂缝自动分类算法[J]. 中国公路学报, 2014, 27(9): 10-18, 24. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL201409003.htm

    PENG Bo, JIANG Yang-sheng, PU Yun. Automated classification algorithm of pavement crack based on digital image processing[J]. China Journal of Highway and Transport, 2014, 27(9): 10-18, 24. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL201409003.htm
    [49]
    吴文周, 李利番, 王结臣. 平面点集凸包Graham算法的改进[J]. 测绘科学, 2010, 35(6): 123-125. https://www.cnki.com.cn/Article/CJFDTOTAL-CHKD201006044.htm

    WU Wen-zhou, LI Li-fan, WANG Jie-chen. An improved Graham algorithm for determining the convex hull of planar points set[J]. Science of Surveying and Mapping, 2010, 35(6): 123-125. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-CHKD201006044.htm
    [50]
    王世兵. 路面裂缝检测系统关键技术的研究与开发[D]. 济南: 济南大学, 2019.

    WANG Shi-bing. Research and development of key technologies for pavement crack detection system[D]. Jinan: University of Jinan, 2019. (in Chinese)
    • Relative Articles
    • Supplements(0)
    • Cited By
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索
    Get Citation
    PDF
    XML

    Article Metrics

    Article views (942) PDF downloads(164) Cited by()
    Proportional views
    Related

    Copyright《Journal of Traffic and Transportation Engineering》编辑部陕ICP备05001904号-1

    Address ：Editorial Department of Journal of Traffic and Transportation Engineering, Chang 'an University, Middle Section of South Second Ring Road, Xi 'an, Shaanxi（710064） Tel：029-82334388 Email：jygc@chd.edu.cn

    All visit:Today's visit:

    Supported by: Beijing Renhe Information Technology Co. Ltd    

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return