多分支特征提取的轻量化路面裂缝分割网络

刘媛媛; 陈嘉豪; 王靖智; 肖乾; 朱路

doi:10.19818/j.cnki.1671-1637.2025.03.013

多分支特征提取的轻量化路面裂缝分割网络

doi: 10.19818/j.cnki.1671-1637.2025.03.013

1.
华东交通大学信息与软件工程学院，江西南昌 330013
2.
华东交通大学载运工具与装备教育部重点实验室，江西南昌 330013

基金项目:

国家自然科学基金项目 62366015

国家自然科学基金项目 61963016

国家自然科学基金项目 61967007

详细信息

作者简介:
刘媛媛(1978-)，女，江西永新人，华东交通大学教授，从事轻量化路面裂缝分割研究

通讯作者:
朱路(1976-)，男，江西永新人，华东交通大学教授，工学博士

中图分类号: U418.66
计量
- 文章访问数: 214
- HTML全文浏览量: 72
- PDF下载量: 17
- 被引次数: 0
出版历程
- 收稿日期: 2024-04-24
- 录用日期: 2025-04-30
- 修回日期: 2025-02-03
- 刊出日期: 2025-06-28

Lightweight pavement crack segmentation network with multi-branch feature extraction

1.
School of Information and Software Engineering, East China Jiaotong University, Nanchang 330013, Jiangxi, China
2.
Key Laboratory of Conveyance and Equipment of Ministry of Education, East China Jiaotong University, Nanchang 330013, Jiangxi, China

Funds:

National Natural Science Foundation of China 62366015

National Natural Science Foundation of China 61963016

National Natural Science Foundation of China 61967007

More Information

Corresponding author: ZHU Lu (1976-), male, professor, PhD, Lzhu@ecjtu.edu.cn

Article Text (Baidu Translation)

摘要

摘要: 针对路面裂缝分割模型难以同时兼顾分割精度与模型复杂度的问题，提出一种轻量化路面裂缝分割网络——分层多分支级联网络(HMBCNet)；根据特征图通道数与模型运算量之间的相关性，采用解耦下采样模块降低网络模型初期的运算量与参数量；利用特征提取中浅层形态信息与深层语义信息的差异性，分别设计多分支扩张模块与双分支反向残差模块对网络不同阶段进行特征提取，多分支的设计能够降低模型的复杂度，同时增强网络模型不同阶段的特征提取能力；通过级联多个子主干网络得到编码器，使特征提取模块的参数能够共享给特征融合模块，实现在不增加模型总参数量的条件下提高模型多尺度特征融合能力；在移动平台NVIDIA Jetson AGX Orin边缘设备上使用自建数据集与公开数据集(Crack500、DeepCrack537)进行试验，采用精确度、召回率、F₁分数、交并比以及准确性作为模型分割性能的评价指标，将参数量、运算量以及运行速度作为模型轻量化程度的评价指标，对比提出的HMBCNet模型与LightCrackNet以及其他7种具有代表性的模型。研究结果表明：在不同场景的数据集中，HMBCNet的分割精度均高于LightCrackNet与其他7种模型，并且大幅度降低参数量与运算量，参数量仅为1.44×10⁶，运算量为2.93 GFLOPs，相较于LightCrackNet运算量降低63%，F₁分数提高1%，在测试阶段平均每张图片的处理时间能够达到211 ms，是一种有效兼顾精度与复杂度且能够应用在实际工程中的网络模型。
- 路面工程 /
- 轻量化分割网络 /
- 分层多分支特征提取 /
- 路面裂缝 /
- 级联融合 /
- 感受野
Abstract: To address the problem in balancing segmentation accuracy and model complexity of road crack segmentation models, a lightweight pavement crack segmentation network, namely, hierarchical multi-branch cascade network (HMBCNet) was proposed. According to the correlation between the number of feature map channels and model computations, a decoupled down sampling module was used to reduce the initial computations and parameters of the network. By exploiting the differences between shallow morphological information and deep semantic information in feature extraction, the multi-branch dilated module and dual-branch inverted residuals module were designed to extract features of the network at different stages. The complexity of the model was reduced and the feature extraction capabilities of the network model at different stages were enhanced by the multi-branch design. Through cascading multiple sub-backbone networks to obtain an encoder, the parameters of the feature extraction module were shared with the feature fusion module. The multi-scale feature fusion capability of the model was thus improved without increasing the total model parameter quantities. Experiments were conducted on the mobile platform NVIDIA Jetson AGX Orin edge device using the self-built dataset and publicly available datasets (Crack500, DeepCrack537). The accuracy, recall, F₁-score, intersection over union, and precision were taken as the evaluation metrics for model segmentation performance. The degree of model lightweight was assessed based on parameter quantities, computation, and running speed. The proposed HMBCNet was compared with LightCrackNet and seven other representative models. Analysis results indicate that across diverse datasets, the segmentation accuracy of HMBCNet is higher than LightCrackNet and the other seven models, and parameter quantities and computations significantly reduce. HMBCNet has only 1.44×10⁶ parameter quantities and 2.93 GFLOPs computations, 63% lower than LightCrackNet in computations and 1% higher in the F₁-score. The average processing time per image during testing reaches 211 ms. HMBCNet proves to be an effective network model that balances accuracy and complexity, suitable for practical engineering applications.
- pavement engineering /
- lightweight segmentation network /
- hierarchical multi-branch feature extraction /
- pavement crack /
- cascade fusion /
- receptive field

HTML全文

图 1 HMBCNet模型结构

Figure 1. HMBCNet model structure

下载: 全尺寸图片幻灯片

图 2 常规下采样模块与解耦下采样模块对比

Figure 2. Comparison between conventional down-sampling module and DDSM

下载: 全尺寸图片幻灯片

图 3 混合扩张模块与多分支扩张模块结构

Figure 3. Structures of hybrid dilated module and MBDM

下载: 全尺寸图片幻灯片

图 4 双分支反向残差模块结构

Figure 4. Structure of DBIRM

下载: 全尺寸图片幻灯片

图 5 级联融合结构

Figure 5. Structure of cascade fusion

下载: 全尺寸图片幻灯片

图 6 级联融合过程与衔接块结构

Figure 6. fusion process and link block structure

下载: 全尺寸图片幻灯片

图 7 自建数据集样本与其对应的标签

Figure 7. Self-built dataset samples and its corresponding labels

下载: 全尺寸图片幻灯片

图 8 Crack500样本与其对应的标签

Figure 8. Crack500 samples and its corresponding labels

下载: 全尺寸图片幻灯片

图 9 DeepCrack537样本与其对应的标签

Figure 9. DeepCrack537 samples and its corresponding labels

下载: 全尺寸图片幻灯片

图 10 各模型在Crack500样本中的测试结果

Figure 10. Test results of each model in Crack500 samples

下载: 全尺寸图片幻灯片

图 11 各模型在DeepCrack537样本中的测试结果

Figure 11. Test results of each model in DeepCrack537 samples

下载: 全尺寸图片幻灯片

图 12 各模型在自建数据集样本中的测试结果

Figure 12. Test results of each model in self-built dataset samples

下载: 全尺寸图片幻灯片

图 13 各模型在Crack500数据集上测试时的运行速度对比

Figure 13. Comparison of running speeds of each model on Crack500 dataset

下载: 全尺寸图片幻灯片

表 1 不同下采样模块的参数量与运算量对比

Table 1. Comparison of parameter quantities and computations of different down-sampling modules

模块	参数量/10³	运算量/GFLOPs
常规下采样模块	56.5	1.89
解耦下采样模块	43.2	0.79

下载: 导出CSV

表 2 不同扩张模块的参数量与运算量对比

Table 2. Comparison of parameter quantities and computations of different dilated modules

模块	参数量/10³	运算量/MFLOPs
混合扩张模块	155.97	639
三分支扩张模块	98.62	404
双分支扩张模块	90.37	370

下载: 导出CSV

表 3 Crack500数据集

Table 3. Crack500 datasets

数据集	训练集/张	验证集/张	测试集/张
Crack500	1 792	448	1 124
Crack500+数据增强	8 960	2 240	1 124

下载: 导出CSV

表 4 各模型在Crack500数据集上的测试指标

Table 4. Test metrics of each model in Crack500 dataset

模型	交并比/%	准确性/%	精确度/%	召回率/%	F₁分数/%
FCN	71.55	96.79	77.44	71.10	71.54
FPN	71.76	96.82	76.28	71.43	71.24
U-Net	71.92	96.84	77.04	71.28	71.78
PSPNet	71.14	96.70	75.68	71.92	70.98
Crack U-Net			65.00	77.20	68.40
FFEDN	58.56	96.95	71.01	76.96	73.87
LightCrackNet	74.50	96.60	75.30	72.10	73.70
HMBCNet	73.16	96.92	76.07	77.31	74.69

下载: 导出CSV

表 5 各模型在DeepCrack537数据集上的测试指标

Table 5. Test metrics of each model in DeepCrack537 dataset

模型	交并比/%	准确性/%	精确度/%	召回率/%	F₁分数/%
FCN	51.11	96.27	46.95	87.99	57.21
FPN	55.08	97.01	51.46	80.81	59.77
U-Net	49.97	95.92	44.21	86.31	54.43
PSPNet	51.63	96.43	46.12	81.41	55.97
HMBCNet	57.67	96.99	52.89	84.52	61.79

下载: 导出CSV

表 6 各模型在自建数据集上的测试指标

Table 6. Test metrics of each model in self-built dataset

模型	交并比/%	准确性/%	精确度/%	召回率/%	F₁分数/%
FCN	47.25	91.08	75.42	56.31	58.16
FPN	49.09	91.28	79.03	53.26	57.96
U-Net	50.60	91.36	71.48	61.96	59.97
PSPNet	30.17	89.32	83.02	22.26	31.07
HMBCNet	56.71	92.53	77.67	65.25	65.91

下载: 导出CSV

表 7 各方法参数量、运算量以及运行速度对比

Table 7. Comparison of parameters, computations and running speeds of each model

模型	参数量/10⁶	运算量/GFLOPs	运行速度/ms
FCN	18.64	80.40	1 340
FPN	23.16	27.48	413
U-Net	24.44	31.36	526
PSPNet	1.61	27.48	229
Crack U-Net	6.00
FFEDN	35.05	262.73
LightCrackNet	1.30	8.00
HMBCNet	1.44	2.93	211

下载: 导出CSV

表 8 HMBCNet中各模块消融结果

Table 8. Ablation results of each module in HMBCNet

数据集	Basic	DDSM	MBDM	DBIRM	CF	交并比/%	准确性/%	精确度/%	召回率/%	F₁分数/%
Crack500	+	-	-	-	-	70.26	96.57	73.05	77.03	72.06
	+	+	-	-	-	70.94	96.64	74.83	75.12	72.26
	+	+	+	-	-	72.72	96.85	74.50	76.69	73.15
	+	+	-	+	-	71.39	96.83	72.43	79.10	73.38
	+	+	+	+	-	73.57	96.97	75.25	76.79	73.82
	+	+	+	+	+	73.16	96.92	76.07	77.31	74.69
DeepCrack537	+	-	-	-	-	52.58	96.08	45.44	86.82	56.10
	+	+	-	-	-	54.21	96.49	47.82	84.96	58.00
	+	+	+	-	-	53.81	96.86	51.15	75.23	57.58
	+	+	-	+	-	53.74	96.17	48.29	84.04	58.03
	+	+	+	+	-	55.54	96.90	53.85	73.28	59.19
	+	+	+	+	+	57.67	96.99	52.89	84.52	61.79

下载: 导出CSV

表 9 HMBCNet结构设计消融结果

Table 9. Ablation results of HMBCNet structure design

数据集	在网络中的顺序			交并比/%	准确性/%	精确度/%	召回率/%	F₁分数/%
数据集	DBIRM	DBDM	TBDM	交并比/%	准确性/%	精确度/%	召回率/%	F₁分数/%
Crack500	1	2	3	73.16	96.92	76.07	77.31	74.69
	2	1	3	72.54	96.82	76.39	75.68	73.85
	3	2	1	70.30	96.63	73.60	76.33	72.17
DeepCrack537	1	2	3	57.67	96.99	52.89	84.52	61.79
	2	1	3	55.73	96.75	50.82	82.91	59.79
	3	2	1	53.77	96.52	47.74	83.83	57.54

下载: 导出CSV

参考文献(33)

[1]	MOHAN A, POOBAL S. Crack detection using image processing: a critical review and analysis[J]. Alexandria Engineering Journal, 2018, 57(2): 787-798.
[2]	易钰程, 王靖智, 朱路, 等. 路面缺陷智能检测系统与方法综述[J]. 华东交通大学学报, 2023, 40(5): 19-31. YI Yu-cheng, WANG Jing-zhi, ZHU Lu, et al. Review of the intelligent pavement defect detection system and methods[J]. Journal of East China Jiaotong University, 2023, 40(5): 19-31.
[3]	张伟光, 钟靖涛, 呼延菊, 等. 基于VGG16-UNet语义分割模型的路面龟裂形态提取与量化[J]. 交通运输工程学报, 2023, 23(2): 166-182. doi: 10.19818/j.cnki.1671-1637.2023.02.012 ZHANG Wei-guang, ZHONG Jing-tao, HUYAN Ju, et al. Extraction and quantification of pavement alligator crack morphology based on VGG16-UNet semantic segmentation model[J]. Journal of Traffic and Transportation Engineering, 2023, 23(2): 166-182. doi: 10.19818/j.cnki.1671-1637.2023.02.012
[4]	SHEN G. Road crack detection based on video image processing[C]//IEEE. 2016 3rd International Conference on Systems and Informatics (ICSAI). New York: IEEE, 2016: 912-917.
[5]	QU Z, CHEN Y X, LIU L, et al. The algorithm of concrete surface crack detection based on the genetic programming and percolation model[J]. IEEE Access, 2019, 7: 57592-57603.
[6]	李海丰, 吴治龙, 聂晶晶, 等. 基于深度图像的机场道面裂缝自动检测算法[J]. 交通运输工程学报, 2020, 20(6): 250-260. doi: 10.19818/j.cnki.1671-1637.2020.06.022 LI Hai-feng, WU Zhi-long, NIE Jing-jing, et al. Automatic crack detection algorithm for airport pavement based on depth image[J]. Journal of Traffic and Transportation Engineering, 2020, 20(6): 250-260. doi: 10.19818/j.cnki.1671-1637.2020.06.022
[7]	田伟, 沈浩, 李晓, 等. 基于图像处理的廊道表面裂缝检测技术研究[J]. 电子设计工程, 2020, 28(5): 148-151, 156. TIAN Wei, SHEN Hao, LI Xiao, et al. Research on corridor surface crack detection technology based on image processing[J]. Electronic Design Engineering, 2020, 28(5): 148-151, 156.
[8]	SHELHAMER E, LONG J, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651.
[9]	LIN T Y, DOLLAR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//IEEE. 2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2017: 2117-2125.
[10]	RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]// Springer. Medical Image Computing and Computer-assisted Intervention-MICCAI 2015. Berlin: Springer, 2015: 234-241.
[11]	祝一帆, 王海涛, 李可, 等. 一种高精度路面裂缝检测网络结构: Crack U-Net[J]. 计算机科学, 2022, 49(1): 204-211. ZHU Yi-fan, WANG Hai-tao, LI Ke, et al. Crack U-Net: towards high quality pavement crack detection[J]. Computer Science, 2022, 49(1): 204-211.
[12]	CHEN L C, ZHU Y K, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Springer. 15th European Conference on Computer Vision (ECCV). Berlin: Springer, 2018: 801-818.
[13]	YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions[J]. arXiv, 2015, DOI: 10.48550/arXiv.1511.07122.
[14]	LIU C Q, ZHU C G, XIA X, et al. FFEDN: feature fusion encoder decoder network for crack detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(9): 15546-15557.
[15]	LIU Z, LI J G, SHEN Z Q, et al. Learning efficient convolutional networks through network slimming[C]//IEEE. 2017 IEEE International Conference on Computer Vision (ICCV). New York: IEEE, 2017: 2736-2744.
[16]	MA X L, LIN S, YE S K, et al. Non-structured DNN weight pruning—Is it beneficial in any platform?[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(9): 4930-4944.
[17]	WANG X J, ZHANG R, SUN Y, et al. Adversarial distillation for learning with privileged provisions[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(3): 786-797.
[18]	ZHAO H S, SHI J P, QI X J, et al. Pyramid scene parsing network[C]//IEEE. 2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2017: 2881-2890.
[19]	WEI H R, LIU X, XU S C, et al. DWRSeg: dilation-wise residual network for real-time semantic segmentation[J]. arXiv, 2022, DOI: 10.48550/arXiv.2212.01173.
[20]	ZHOU Q, QU Z, JU F R. A lightweight network for crack detection with split exchange convolution and multi-scale features fusion[J]. IEEE Transactions on Intelligent Vehicles, 2023, 8(3): 2296-2306.
[21]	ZHANG G, LI Z Y, LI J M, et al. CFNet: cascade fusion network for dense prediction[J]. arXiv, 2023, DOI: 10.48550/arXiv.2302.06052.
[22]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Springer. 15th European Conference on Computer Vision (ECCV). Berlin: Springer, 2018: 3-19.
[23]	WANG W G, SUN G L, VAN GOOL L. Looking beyond single images for weakly supervised semantic segmentation learning[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, 46(3): 1635-1649.
[24]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// IEEE. 2016 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2016: 770-778.
[25]	HOWARD A G, ZHU M L, CHEN B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications[J]. arXiv, 2017, DOI: 10.48550/arXiv.1704.04861.
[26]	SANDLER M, HOWARD A, ZHU M L, et al. MobileNetv2: Inverted residuals and linear bottlenecks[C]//IEEE. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2018: 4510-4520.
[27]	HU M, LI Y L, FANG L, et al. A2-FPN: attention aggregation based feature pyramid network for instance segmentation[C]//IEEE. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2021: 15343-15352.
[28]	MENG T J, GHIASI G, MAHJOURIAN R, et al. Revisiting multi-scale feature fusion for semantic segmentation[J]. arXiv, 2022, DOI: 10.48550/arXiv.2203.12683.
[29]	翟军治, 孙朝云, 裴莉莉, 等. 多尺度特征增强的路面裂缝检测方法[J]. 交通运输工程学报, 2023, 23(1): 291-308. doi: 10.19818/j.cnki.1671-1637.2023.01.022 ZHAI Jun-zhi, SUN Zhao-yun, PEI Li-li, et al. Pavement crack detection method based on multi-scale feature enhancement[J]. Journal of Traffic and Transportation Engineering, 2023, 23(1): 291-308. doi: 10.19818/j.cnki.1671-1637.2023.01.022
[30]	LI X, SUN X, MENG Y, et al. Dice loss for data-imbalanced NLP tasks[J]. arXiv, 2019, DOI: 10.48550/arXiv.1911.02855.
[31]	SMITH L N, TOPIN N. Super-convergence: very fast training of neural networks using large learning rates[J]. arXiv, 2017, DOI: 10.48550/arXiv.1708.07120.
[32]	YANG F, ZHANG L, YU S J, et al. Feature pyramid and hierarchical boosting network for pavement crack detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2019, 21(4): 1525-1535.
[33]	LIU Y H, YAO J, LU X H, et al. DeepCrack: a deep hierarchical feature learning architecture for crack segmentation[J]. Neurocomputing, 2019, 338: 139-153.