留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

多维图像融合驱动的复杂场景路表破损识别

管进超 丁玲 杨旭 刘鹏飞 汪海年

管进超, 丁玲, 杨旭, 刘鹏飞, 汪海年. 多维图像融合驱动的复杂场景路表破损识别[J]. 交通运输工程学报, 2024, 24(3): 154-170. doi: 10.19818/j.cnki.1671-1637.2024.03.010
引用本文: 管进超, 丁玲, 杨旭, 刘鹏飞, 汪海年. 多维图像融合驱动的复杂场景路表破损识别[J]. 交通运输工程学报, 2024, 24(3): 154-170. doi: 10.19818/j.cnki.1671-1637.2024.03.010
GUAN Jin-chao, DING Ling, YANG Xu, LIU Peng-fei, WANG Hai-nian. Pavement surface distress detection in complex scenarios driven by multi-dimensional image fusion[J]. Journal of Traffic and Transportation Engineering, 2024, 24(3): 154-170. doi: 10.19818/j.cnki.1671-1637.2024.03.010
Citation: GUAN Jin-chao, DING Ling, YANG Xu, LIU Peng-fei, WANG Hai-nian. Pavement surface distress detection in complex scenarios driven by multi-dimensional image fusion[J]. Journal of Traffic and Transportation Engineering, 2024, 24(3): 154-170. doi: 10.19818/j.cnki.1671-1637.2024.03.010

多维图像融合驱动的复杂场景路表破损识别

doi: 10.19818/j.cnki.1671-1637.2024.03.010
基金项目: 

国家重点研发计划 2021YFB2601000

国家自然科学基金项目 52078049

详细信息
    作者简介:

    管进超(1995-),男,江苏常州人,浙江沪杭甬高速公路股份有限公司博士后,从事道路检测与养护研究

    通讯作者:

    丁玲(1986-),女,江苏盐城人,长安大学讲师,工学博士

  • 中图分类号: U418.6

Pavement surface distress detection in complex scenarios driven by multi-dimensional image fusion

Funds: 

National Key Research and Development Program of China 2021YFB2601000

National Natural Science Foundation of China 52078049

More Information
  • 摘要: 为提升复杂场景中路表裂缝与坑槽的识别精度和鲁棒性,考虑实际检测场景中路表破损形态的不规则性和环境噪声干扰,提出了一种面向多维图像的路表破损自动分割模型与特征融合优化方法;基于多目立体视觉重构的路表高精度点云模型,通过同源点云栅格化生成二、三维图像,建立了复杂场景路表破损图像数据集;结合深度可分离卷积和多层位特征叠加,构造了轻量化编码-解码网络PDU-net,用于像素级裂缝与坑槽识别;在分割模型基础上,提出了像素运算和通道重组2种多维图像融合策略,以提升深度学习网络对浅细裂缝特征的提取效率。试验结果表明:PDU-net模型能够有效学习不同类型图像和破损特征,在不同数据集上的训练损失均能稳定收敛,其中三维图像训练周期小于二维图像;相较于现有卷积分割网络,PDU-net模型在复杂场景下的路表破损分割精度和效率更高,三维裂缝与坑槽图像分割的调和均值分别为81.00%和95.85%,平均正向推理时间约为现有模型的30%;多维融合图像可以提升复杂裂缝分割的精度和鲁棒性,在最优色彩-深度比为0.2时,裂缝分割的调和均值可提升至83.31%。综上所述,所提出的方法可在复杂场景中有效抑制环境噪声并强化病害特征。

     

  • 图  1  路表点云栅格化

    Figure  1.  Rasterization of pavement surface point clouds

    图  2  不同类型与严重程度的三维破损图像

    Figure  2.  3D distress images with different types and severities

    图  3  不同环境噪声下的二维破损图像

    Figure  3.  2D distress images with different environmental noises

    图  4  深度可分离卷积运算

    Figure  4.  Operation of depthwise separable convolution

    图  5  轻量化编码-解码网络框架

    Figure  5.  Framework of lightweight encoding-decoding network

    图  6  基于像素运算的多维融合图像生成

    Figure  6.  Multi-dimensional fusion image generation based on pixel operation

    图  7  基于通道重组的多维融合图像生成

    Figure  7.  Multi-dimensional fusion image generation based on channel recombination

    图  8  不同PDU-net模型训练过程

    Figure  8.  Training processes of different PDU-net models

    图  9  路表破损图像分割定量评价

    Figure  9.  Quantitative evaluation of pavement surface distress image segmentation

    图  10  二维裂缝图像分割可视化

    Figure  10.  Visualization of 2D crack image segmentation

    图  11  三维裂缝图像分割可视化

    Figure  11.  Visualization of 3D crack image segmentation

    图  12  二维坑槽图像分割可视化

    Figure  12.  Visualization of 2D pothole image segmentation

    图  13  三维坑槽图像分割可视化

    Figure  13.  Visualization of 3D pothole image segmentation

    图  14  像素运算多维融合图像分割性能定量评价

    Figure  14.  Quantitative evaluation of multi-dimensional fusion image segmentation with pixel operation

    图  15  不同裂缝图像分割精度分布

    Figure  15.  Distributions of different crack image segmentation accuracies

    图  16  通道重组多维融合图像分割性能定量评价

    Figure  16.  Quantitative evaluation of multi-dimensional fusion image segmentation with channel recombination

    图  17  多维融合裂缝图像分割可视化

    Figure  17.  Visualization of multi-dimensional fusion image segmentation

    表  1  不同模型的计算效率

    Table  1.   Computational efficiencies of different models

    模型名称 模型参数量/106 模型体积/MB 推理速度/s 每秒处理图像张数
    PDU-net 6.3 24.3 0.038 26.3
    GCU-net 10.3 39.4 0.133 7.5
    U-net 10.1 38.8 0.122 8.2
    下载: 导出CSV
  • [1] 中华人民共和国交通运输部. 2021年交通运输行业发展统计公报[R]. 北京: 中华人民共和国交通运输部, 2022.

    Ministry of Transport of the People's Republic of China. Statistical bulletin on transportation industry development in 2022[R]. Beijing: Ministry of Transport of the People's Republic of China, 2022. (in Chinese)
    [2] YANG Xu, GUAN Jin-chao, DING Ling, et al. Research and applications of artificial neural network in pavement engineering: a state-of-the-art review[J]. Journal of Traffic and Transportation Engineering (English Edition), 2021, 8(6): 1000-1021. doi: 10.1016/j.jtte.2021.03.005
    [3] 徐志刚, 车艳丽, 李金龙, 等. 路面破损图像自动处理技术研究进展[J]. 交通运输工程学报, 2019, 19(1): 172-190. doi: 10.3969/j.issn.1671-1637.2019.01.017

    XU Zhi-gang, CHE Yan-li, LI Jin-long, et al. Research progress on automatic image processing technology for pavement distress[J]. Journal of Traffic and Transportation Engineering, 2019, 19(1): 172-190. (in Chinese) doi: 10.3969/j.issn.1671-1637.2019.01.017
    [4] ZHANG Ce, NATEGHINIA E, MIRANDA-MORENO L F, et al. Pavement distress detection using convolutional neural network (CNN): a case study in Montreal, Canada[J]. International Journal of Transportation Science and Technology, 2022, 11(2): 298-309. doi: 10.1016/j.ijtst.2021.04.008
    [5] 李清泉, 胡庆武. 基于图像自动匀光的路面裂缝图像分析方法[J]. 公路交通科技, 2010, 27(4): 1-5, 27. doi: 10.3969/j.issn.1002-0268.2010.04.001

    LI Qing-quan, HU Qing-wu. A pavement crack image analysis approach based on automatic image dodging[J]. Journal of Highway and Transportation Research and Development, 2010, 27(4): 1-5, 27. (in Chinese) doi: 10.3969/j.issn.1002-0268.2010.04.001
    [6] 孙朝云, 赵海伟, 李伟, 等. 基于双相扫描检测的路面三维裂缝识别方法[J]. 中国公路学报, 2015, 28(2): 26-32. doi: 10.3969/j.issn.1001-7372.2015.02.004

    SUN Zhao-yun, ZHAO Hai-wei, LI Wei, et al. 3D pavement crack identification method based on dual-phase scanning detection[J]. China Journal of Highway and Transport, 2015, 28(2): 26-32. (in Chinese) doi: 10.3969/j.issn.1001-7372.2015.02.004
    [7] JO Y, RYU S K, KIM Y R. Pothole detection based on the features of intensity and motion[J]. Transportation Research Record, 2016, 2595(1): 18-28. doi: 10.3141/2595-03
    [8] SOLLAZZO G, WANG K C P, BOSURGI G, et al. Hybrid procedure for automated detection of cracking with 3D pavement data[J]. Journal of Computing in Civil Engineering, 2016, 30(6): 04016032. doi: 10.1061/(ASCE)CP.1943-5487.0000597
    [9] 尹冠生, 高建国, 史明辉, 等. 图像分块下的隧道裂缝识别方法[J]. 交通运输工程学报, 2022, 22(2): 148-159. doi: 10.19818/j.cnki.1671-1637.2022.02.011

    YIN Guan-sheng, GAO Jian-guo, SHI Ming-hui, et al. Tunnel crack recognition method under image block[J]. Journal of Traffic and Transportation Engineering, 2022, 22(2): 148-159. (in Chinese) doi: 10.19818/j.cnki.1671-1637.2022.02.011
    [10] PARK S, BANG S, KIM H, et al. Patch-based crack detection in black box images using convolutional neural networks[J]. Journal of Computing in Civil Engineering, 2019, 33(3): 04019017. doi: 10.1061/(ASCE)CP.1943-5487.0000831
    [11] HUYAN Ju, LI Wei, TIGHE S, et al. Detection of sealed and unsealed cracks with complex backgrounds using deep convolutional neural network[J]. Automation in Construction, 2019, 107: 102946. doi: 10.1016/j.autcon.2019.102946
    [12] 张志华, 邓砚学, 张新秀. 基于改进SegNet的沥青路面病害提取与分类方法[J]. 交通信息与安全, 2022, 40(3): 127-135. doi: 10.3963/j.jssn.1674-4861.2022.03.013

    ZHANG Zhi-hua, DENG Yan-xue, ZHANG Xin-xiu. A method for detecting and differentiating asphalt pavement distress based on an improved SegNet algorithm[J]. Journal of Transport Information and Safety, 2022, 40(3): 127-135. (in Chinese) doi: 10.3963/j.jssn.1674-4861.2022.03.013
    [13] YANG Fan, ZHANG Lei, YU Si-jia, et al. Feature pyramid and hierarchical boosting network for pavement crack detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 21(4): 1525-1535. doi: 10.1109/TITS.2019.2910595
    [14] TONG Zheng, YUAN Dong-dong, GAO Jie, et al. Pavement defect detection with fully convolutional network and an uncertainty framework[J]. Computer-Aided Civil and Infrastructure Engineering, 2020, 35(8): 832-849. doi: 10.1111/mice.12533
    [15] HUYAN Ju, LI Wei, TIGHE S, et al. CrackU-net: a novel deep convolutional neural network for pixelwise pavement crack detection[J]. Structural Control and Health Monitoring, 2020, 27(8): e2551.
    [16] CHEN Han-shen, LIN Hui-ping, YAO Ming-hai. Improving the efficiency of encoder-decoder architecture for pixel-level crack detection[J]. IEEE Access, 2019, 7: 186657-186670. doi: 10.1109/ACCESS.2019.2961375
    [17] MATHAVAN S, KAMAL K, RAHMAN M. A review of three-dimensional imaging technologies for pavement distress detection and measurements[J]. IEEE Transactions on Intelligent Transportation Systems, 2015, 16(5): 2353-2362. doi: 10.1109/TITS.2015.2428655
    [18] CAO Wen-ming, LIU Qi-fan, HE Zhi-quan. Review of pavement defect detection methods[J]. IEEE Access, 2020, 8: 14531-14544. doi: 10.1109/ACCESS.2020.2966881
    [19] ZHANG De-jin, ZOU Qin, LIN Hong, et al. Automatic pavement defect detection using 3D laser profiling technology[J]. Automation in Construction, 2018, 96: 350-365. doi: 10.1016/j.autcon.2018.09.019
    [20] 丁世海, 战友, 阳恩慧, 等. 基于高精度激光断面高程的沥青路面MTD测量[J]. 东南大学学报(自然科学版), 2020, 50(1): 137-142. https://www.cnki.com.cn/Article/CJFDTOTAL-DNDX202001018.htm

    DING Shi-hai, ZHAN You, YANG En-hui, et al. MTD measurement of asphalt pavement based on high precision laser section elevation[J]. Journal of Southeast University (Natural Science Edition), 2020, 50(1): 137-142. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-DNDX202001018.htm
    [21] CHEN Jia-ying, HUANG Xiao-ming, ZHENG Bin-shuang, et al. Real-time identification system of asphalt pavement texture based on the close-range photogrammetry[J]. Construction and Building Materials, 2019, 226: 910-919. doi: 10.1016/j.conbuildmat.2019.07.321
    [22] ZHANG A, WANG K C P, LI Bao-xian, et al. Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network[J]. Computer-Aided Civil and Infrastructure Engineering, 2017, 32(10): 805-819. doi: 10.1111/mice.12297
    [23] ZHANG A, WANG K C P, FEI Yue, et al. Deep learning-based fully automated pavement crack detection on 3D asphalt surfaces with an improved CrackNet[J]. Journal of Computing in Civil Engineering, 2018, 32(5): 04018041. doi: 10.1061/(ASCE)CP.1943-5487.0000775
    [24] ZHANG A, WANG K C P, FEI Yue, et al. Automated pixel-level pavement crack detection on 3D asphalt surfaces with a recurrent neural network[J]. Computer-Aided Civil and Infrastructure Engineering, 2019, 34(3): 213-229. doi: 10.1111/mice.12409
    [25] FEI Yue, WANG K C P, ZHANG A, et al. Pixel-level cracking detection on 3D asphalt pavement images through deep-learning-based CrackNet-V[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 21(1): 273-284. doi: 10.1109/TITS.2019.2891167
    [26] GUAN Jin-chao, YANG Xu, DING Ling, et al. Automated pixel-level pavement distress detection based on stereo vision and deep learning[J]. Automation in Construction, 2021, 129: 103788. doi: 10.1016/j.autcon.2021.103788
    [27] 曾清红, 卢德唐. 基于移动最小二乘法的曲线曲面拟合[J]. 工程图学学报, 2004, 25(1): 84-89. doi: 10.3969/j.issn.1003-0158.2004.01.017

    ZENG Qing-hong, LU De-tang. Curve and surface fitting based on moving least-squares methods[J]. Journal of Graphics, 2004, 25(1): 84-89. (in Chinese) doi: 10.3969/j.issn.1003-0158.2004.01.017
    [28] HOWARD A G, ZHU M, CHEN B, et al. Mobilenets: efficient convolutional neural networks for mobile vision applications[J]. arXiv, 2017, DOI: 10.48550/arXiv.1704.04861.
    [29] RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C]// NAVAB N, HORNEGGER J, WELLS W, et al. 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich: Springer, 2015: 234-241.
    [30] LAU S L H, CHONG E K P, YANG X, et al. Automated pavement crack segmentation using U-Net-based convolutional neural network[J]. IEEE Access, 2020, 8: 114892. doi: 10.1109/ACCESS.2020.3003638
    [31] CHEN Jie, LIU Gang, CHEN Xin. Road crack image segmentation using global context U-net[C]//GOKHALE A, TAN Y. Proceedings of the 2019 3rd International Conference on Computer Science and Artificial Intelligence. New York: Association for Computing Machinery, 2019: 181-185.
  • 加载中
图(17) / 表(1)
计量
  • 文章访问数:  196
  • HTML全文浏览量:  46
  • PDF下载量:  52
  • 被引次数: 0
出版历程
  • 收稿日期:  2024-01-19
  • 网络出版日期:  2024-07-18
  • 刊出日期:  2024-06-30

目录

    /

    返回文章
    返回