Volume 24 Issue 3
Jun.  2024
Turn off MathJax
Article Contents
Article Navigation >  Journal of Traffic and Transportation Engineering  > 2024  >  24(3): 154-170
GUAN Jin-chao, DING Ling, YANG Xu, LIU Peng-fei, WANG Hai-nian. Pavement surface distress detection in complex scenarios driven by multi-dimensional image fusion[J]. Journal of Traffic and Transportation Engineering, 2024, 24(3): 154-170. doi: 10.19818/j.cnki.1671-1637.2024.03.010
Citation: GUAN Jin-chao, DING Ling, YANG Xu, LIU Peng-fei, WANG Hai-nian. Pavement surface distress detection in complex scenarios driven by multi-dimensional image fusion[J]. Journal of Traffic and Transportation Engineering, 2024, 24(3): 154-170. doi: 10.19818/j.cnki.1671-1637.2024.03.010
shu

Pavement surface distress detection in complex scenarios driven by multi-dimensional image fusion

doi: 10.19818/j.cnki.1671-1637.2024.03.010
  • 1.

    School of Highway, Chang'an University, Xi'an 710064, Shaanxi, China

  • 2.

    Zhejiang Expressway Co., Ltd., Hangzhou 310020, Zhejiang, China

  • 3.

    School of Transportation Engineering, Chang'an University, Xi'an 710064, Shaanxi, China

  • 4.

    School of Future Transportation, Chang'an University, Xi'an 710064, Shaanxi, China

  • 5.

    Institute of Highway Engineering, RWTH Aachen University, Aachen 52074, North Rhine-Westphalia, Germany

Funds:

National Key Research and Development Program of China 2021YFB2601000

National Natural Science Foundation of China 52078049

More Information
  • Author Bio:

    GUAN Jin-chao(1995-), male, postdoctor, guan@chd.edu.cn

    DING Ling(1986-), female, assistant professor, PhD, dingling@chd.edu.cn

  • Received Date: 2024-01-19
    Available Online: 2024-07-18
  • Publish Date: 2024-06-30
    Abstract
  • To improve the accuracy and robustness of crack and pothole detection of pavement surface in complex scenarios, the morphological irregularity of pavement surface distresses and the influence of environmental noises in practical detection scenarios were considered, and an automatic pavement surface distress segmentation model and feature fusion optimization method for multi-dimensional images were proposed. Based on high-precision pavement surface point cloud models reconstructed by multi-view stereo vision, 2D and 3D images were generated by the rasterization of homologous point clouds. The pavement surface distress image dataset in complex scenarios was established. A lightweight encoding-decoding network, namely PDU-net, integrating depthwise separable convolution and multi-layer feature combination, was developed for pixel-level crack and pothole detection. Based on the segmentation model, two multi-dimensional image fusion strategies, including pixel operation and channel recombination, were proposed to improve the extraction efficiency of deep learning networks in shallow and fine crack features. Experimental results show that the PDU-net model can effectively learn features from different types of images and distresses. The training loss of the PDU-net on different datasets can converge stably, with the training cycles of 3D images shorter than that of 2D images. Compared with existing convolutional segmentation networks, the PDU-net model achieves higher accuracy and efficiency for pavement surface distress segmentation in complex scenarios. The harmonic means of 3D crack and pothole image segmentation are 81.00% and 95.85%, respectively. The average forward inference time of the PDU-net is about 30% of the existing models. The segmentation accuracy and robustness of complex cracks can be improved by multi-dimensional fusion images. When the optimal color-depth ratio is 0.2, the harmonic mean of the crack segmentation increases to 83.31%. In conclusion, the proposed method can effectively suppress environmental noises and strengthen the surface distress features in complex scenarios.

     

    • FullText(HTML)
  • loading
    • References(31)
  • [1]
    中华人民共和国交通运输部. 2021年交通运输行业发展统计公报[R]. 北京: 中华人民共和国交通运输部, 2022.

    Ministry of Transport of the People's Republic of China. Statistical bulletin on transportation industry development in 2022[R]. Beijing: Ministry of Transport of the People's Republic of China, 2022. (in Chinese)
    [2]
    YANG Xu, GUAN Jin-chao, DING Ling, et al. Research and applications of artificial neural network in pavement engineering: a state-of-the-art review[J]. Journal of Traffic and Transportation Engineering (English Edition), 2021, 8(6): 1000-1021. doi: 10.1016/j.jtte.2021.03.005
    [3]
    徐志刚, 车艳丽, 李金龙, 等. 路面破损图像自动处理技术研究进展[J]. 交通运输工程学报, 2019, 19(1): 172-190. doi: 10.3969/j.issn.1671-1637.2019.01.017

    XU Zhi-gang, CHE Yan-li, LI Jin-long, et al. Research progress on automatic image processing technology for pavement distress[J]. Journal of Traffic and Transportation Engineering, 2019, 19(1): 172-190. (in Chinese) doi: 10.3969/j.issn.1671-1637.2019.01.017
    [4]
    ZHANG Ce, NATEGHINIA E, MIRANDA-MORENO L F, et al. Pavement distress detection using convolutional neural network (CNN): a case study in Montreal, Canada[J]. International Journal of Transportation Science and Technology, 2022, 11(2): 298-309. doi: 10.1016/j.ijtst.2021.04.008
    [5]
    李清泉, 胡庆武. 基于图像自动匀光的路面裂缝图像分析方法[J]. 公路交通科技, 2010, 27(4): 1-5, 27. doi: 10.3969/j.issn.1002-0268.2010.04.001

    LI Qing-quan, HU Qing-wu. A pavement crack image analysis approach based on automatic image dodging[J]. Journal of Highway and Transportation Research and Development, 2010, 27(4): 1-5, 27. (in Chinese) doi: 10.3969/j.issn.1002-0268.2010.04.001
    [6]
    孙朝云, 赵海伟, 李伟, 等. 基于双相扫描检测的路面三维裂缝识别方法[J]. 中国公路学报, 2015, 28(2): 26-32. doi: 10.3969/j.issn.1001-7372.2015.02.004

    SUN Zhao-yun, ZHAO Hai-wei, LI Wei, et al. 3D pavement crack identification method based on dual-phase scanning detection[J]. China Journal of Highway and Transport, 2015, 28(2): 26-32. (in Chinese) doi: 10.3969/j.issn.1001-7372.2015.02.004
    [7]
    JO Y, RYU S K, KIM Y R. Pothole detection based on the features of intensity and motion[J]. Transportation Research Record, 2016, 2595(1): 18-28. doi: 10.3141/2595-03
    [8]
    SOLLAZZO G, WANG K C P, BOSURGI G, et al. Hybrid procedure for automated detection of cracking with 3D pavement data[J]. Journal of Computing in Civil Engineering, 2016, 30(6): 04016032. doi: 10.1061/(ASCE)CP.1943-5487.0000597
    [9]
    尹冠生, 高建国, 史明辉, 等. 图像分块下的隧道裂缝识别方法[J]. 交通运输工程学报, 2022, 22(2): 148-159. doi: 10.19818/j.cnki.1671-1637.2022.02.011

    YIN Guan-sheng, GAO Jian-guo, SHI Ming-hui, et al. Tunnel crack recognition method under image block[J]. Journal of Traffic and Transportation Engineering, 2022, 22(2): 148-159. (in Chinese) doi: 10.19818/j.cnki.1671-1637.2022.02.011
    [10]
    PARK S, BANG S, KIM H, et al. Patch-based crack detection in black box images using convolutional neural networks[J]. Journal of Computing in Civil Engineering, 2019, 33(3): 04019017. doi: 10.1061/(ASCE)CP.1943-5487.0000831
    [11]
    HUYAN Ju, LI Wei, TIGHE S, et al. Detection of sealed and unsealed cracks with complex backgrounds using deep convolutional neural network[J]. Automation in Construction, 2019, 107: 102946. doi: 10.1016/j.autcon.2019.102946
    [12]
    张志华, 邓砚学, 张新秀. 基于改进SegNet的沥青路面病害提取与分类方法[J]. 交通信息与安全, 2022, 40(3): 127-135. doi: 10.3963/j.jssn.1674-4861.2022.03.013

    ZHANG Zhi-hua, DENG Yan-xue, ZHANG Xin-xiu. A method for detecting and differentiating asphalt pavement distress based on an improved SegNet algorithm[J]. Journal of Transport Information and Safety, 2022, 40(3): 127-135. (in Chinese) doi: 10.3963/j.jssn.1674-4861.2022.03.013
    [13]
    YANG Fan, ZHANG Lei, YU Si-jia, et al. Feature pyramid and hierarchical boosting network for pavement crack detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 21(4): 1525-1535. doi: 10.1109/TITS.2019.2910595
    [14]
    TONG Zheng, YUAN Dong-dong, GAO Jie, et al. Pavement defect detection with fully convolutional network and an uncertainty framework[J]. Computer-Aided Civil and Infrastructure Engineering, 2020, 35(8): 832-849. doi: 10.1111/mice.12533
    [15]
    HUYAN Ju, LI Wei, TIGHE S, et al. CrackU-net: a novel deep convolutional neural network for pixelwise pavement crack detection[J]. Structural Control and Health Monitoring, 2020, 27(8): e2551.
    [16]
    CHEN Han-shen, LIN Hui-ping, YAO Ming-hai. Improving the efficiency of encoder-decoder architecture for pixel-level crack detection[J]. IEEE Access, 2019, 7: 186657-186670. doi: 10.1109/ACCESS.2019.2961375
    [17]
    MATHAVAN S, KAMAL K, RAHMAN M. A review of three-dimensional imaging technologies for pavement distress detection and measurements[J]. IEEE Transactions on Intelligent Transportation Systems, 2015, 16(5): 2353-2362. doi: 10.1109/TITS.2015.2428655
    [18]
    CAO Wen-ming, LIU Qi-fan, HE Zhi-quan. Review of pavement defect detection methods[J]. IEEE Access, 2020, 8: 14531-14544. doi: 10.1109/ACCESS.2020.2966881
    [19]
    ZHANG De-jin, ZOU Qin, LIN Hong, et al. Automatic pavement defect detection using 3D laser profiling technology[J]. Automation in Construction, 2018, 96: 350-365. doi: 10.1016/j.autcon.2018.09.019
    [20]
    丁世海, 战友, 阳恩慧, 等. 基于高精度激光断面高程的沥青路面MTD测量[J]. 东南大学学报(自然科学版), 2020, 50(1): 137-142. https://www.cnki.com.cn/Article/CJFDTOTAL-DNDX202001018.htm

    DING Shi-hai, ZHAN You, YANG En-hui, et al. MTD measurement of asphalt pavement based on high precision laser section elevation[J]. Journal of Southeast University (Natural Science Edition), 2020, 50(1): 137-142. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-DNDX202001018.htm
    [21]
    CHEN Jia-ying, HUANG Xiao-ming, ZHENG Bin-shuang, et al. Real-time identification system of asphalt pavement texture based on the close-range photogrammetry[J]. Construction and Building Materials, 2019, 226: 910-919. doi: 10.1016/j.conbuildmat.2019.07.321
    [22]
    ZHANG A, WANG K C P, LI Bao-xian, et al. Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network[J]. Computer-Aided Civil and Infrastructure Engineering, 2017, 32(10): 805-819. doi: 10.1111/mice.12297
    [23]
    ZHANG A, WANG K C P, FEI Yue, et al. Deep learning-based fully automated pavement crack detection on 3D asphalt surfaces with an improved CrackNet[J]. Journal of Computing in Civil Engineering, 2018, 32(5): 04018041. doi: 10.1061/(ASCE)CP.1943-5487.0000775
    [24]
    ZHANG A, WANG K C P, FEI Yue, et al. Automated pixel-level pavement crack detection on 3D asphalt surfaces with a recurrent neural network[J]. Computer-Aided Civil and Infrastructure Engineering, 2019, 34(3): 213-229. doi: 10.1111/mice.12409
    [25]
    FEI Yue, WANG K C P, ZHANG A, et al. Pixel-level cracking detection on 3D asphalt pavement images through deep-learning-based CrackNet-V[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 21(1): 273-284. doi: 10.1109/TITS.2019.2891167
    [26]
    GUAN Jin-chao, YANG Xu, DING Ling, et al. Automated pixel-level pavement distress detection based on stereo vision and deep learning[J]. Automation in Construction, 2021, 129: 103788. doi: 10.1016/j.autcon.2021.103788
    [27]
    曾清红, 卢德唐. 基于移动最小二乘法的曲线曲面拟合[J]. 工程图学学报, 2004, 25(1): 84-89. doi: 10.3969/j.issn.1003-0158.2004.01.017

    ZENG Qing-hong, LU De-tang. Curve and surface fitting based on moving least-squares methods[J]. Journal of Graphics, 2004, 25(1): 84-89. (in Chinese) doi: 10.3969/j.issn.1003-0158.2004.01.017
    [28]
    HOWARD A G, ZHU M, CHEN B, et al. Mobilenets: efficient convolutional neural networks for mobile vision applications[J]. arXiv, 2017, DOI: 10.48550/arXiv.1704.04861.
    [29]
    RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C]// NAVAB N, HORNEGGER J, WELLS W, et al. 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich: Springer, 2015: 234-241.
    [30]
    LAU S L H, CHONG E K P, YANG X, et al. Automated pavement crack segmentation using U-Net-based convolutional neural network[J]. IEEE Access, 2020, 8: 114892. doi: 10.1109/ACCESS.2020.3003638
    [31]
    CHEN Jie, LIU Gang, CHEN Xin. Road crack image segmentation using global context U-net[C]//GOKHALE A, TAN Y. Proceedings of the 2019 3rd International Conference on Computer Science and Artificial Intelligence. New York: Association for Computing Machinery, 2019: 181-185.
    • Relative Articles
    • Supplements(0)
    • Cited By
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索
    Get Citation
    PDF
    XML

    Article Metrics

    Article views (289) PDF downloads(54) Cited by()
    Proportional views
    Related

    Copyright《Journal of Traffic and Transportation Engineering》编辑部陕ICP备05001904号-1

    Address ：Editorial Department of Journal of Traffic and Transportation Engineering, Chang 'an University, Middle Section of South Second Ring Road, Xi 'an, Shaanxi（710064） Tel：029-82334388 Email：jygc@chd.edu.cn

    All visit:Today's visit:

    Supported by: Beijing Renhe Information Technology Co. Ltd    

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return