Pavement crack detection method based on multi-scale feature enhancement

ZHAI Jun-zhi; SUN Zhao-yun; PEI Li-li; HUYAN Ju; LI Wei

doi:10.19818/j.cnki.1671-1637.2023.01.022

Volume 23 Issue 1

Feb. 2023

Turn off MathJax

Article Contents

Article Navigation > Journal of Traffic and Transportation Engineering > 2023 > 23(1): 291-308

ZHAI Jun-zhi, SUN Zhao-yun, PEI Li-li, HUYAN Ju, LI Wei. Pavement crack detection method based on multi-scale feature enhancement[J]. Journal of Traffic and Transportation Engineering, 2023, 23(1): 291-308. doi: 10.19818/j.cnki.1671-1637.2023.01.022

Citation:

ZHAI Jun-zhi, SUN Zhao-yun, PEI Li-li, HUYAN Ju, LI Wei. Pavement crack detection method based on multi-scale feature enhancement[J]. Journal of Traffic and Transportation Engineering, 2023, 23(1): 291-308. doi: 10.19818/j.cnki.1671-1637.2023.01.022

Citation:

PDF( 34671 KB)

Pavement crack detection method based on multi-scale feature enhancement

doi: 10.19818/j.cnki.1671-1637.2023.01.022

1.
School of Information Engineering, Chang'an University, Xi'an 710064, Shaanxi, China
2.
School of Transportation, Southeast University, Nanjing 211189, Jiangsu, China

Funds:

National Key Research and Development Program of China 2021YFB1600205

National Natural Science Foundation of China 52178407

National Natural Science Foundation of China 51978071

Key Research and Development Program of Shaanxi Province 2022JBGS3-08

Fundamental Research Funds for the Central Universities 300102242901

More Information

Author Bio:
ZHAI Jun-zhi(1981-), male, doctoral student, zjz5250@sina.cn

SUN Zhao-yun(1962-), female, professor, PhD, zhaoyunsun@126.com
Received Date: 2022-09-02
Available Online: 2023-03-08
Publish Date: 2023-02-25

Abstract

Abstract

To solve the problems of incomplete pavement crack detection and discontinuous segmentation, a detection network MFENet for pavement cracks based on multi-scale feature enhancement was proposed, and the detection, classification and segmentation of end-to-end pavement crack images were realized. A multi-scale attention-based feature enhancement module was designed, and the mapping relationships of the weight coefficients of the upper multi-scale feature channels with those of the lower feature channels in the network model were determined to highlight the feature outputs from the effective channels. Based on the correlation between the coordinate information of the pavement crack and the semantic information of the pixels in physical location, a multi-semantic feature correlation module was designed and thereby feature fusion and enhancement among different semantic information were achieved. Then, the foreground features of the pavement crack image were filtered by feature dimension transformation. A quantitative evaluation method for deep feature intensity was proposed to improve the interpretability of the model's feature extraction ability. Research results on self-collected dataset show that the average precision and average recall of the MFENet in pavement crack image detection are 4.3% and 5.4% higher than those of the Mask R-CNN, respectively, and 14.6% and 14.3% higher than those of the baseline model RDSNet, respectively. The average precision and average recall of the MFENet in pavement crack image segmentation are 6.6% and 8.8% higher than those of the Mask R-CNN, respectively, and 8.1% and 9.7% higher than those of the RDSNet, respectively. In the comparison with the Mask R-CNN and other mainstream methods, the images of different types of pavement cracks are detected and segmented with the highest accuracy by the MFENet. Research results on public datasets (CFD and CRACK500) show that the detection and segmentation accuracy of the MFENet are invariably higher than those of the Mask R-CNN and other mainstream methods on the datasets covering different scenarios, indicating the higher robustness of the proposed method. In addition, the processing speed of the MFENet is also faster than that of the RDSNet on different datasets.
- pavement crack detection,
- multi-scale attention,
- feature enhancement,
- multi-semantic,
- interpretability,
- robustness

FullText(HTML)

References(32)

References

[1]	CAO Wen-ming, LIU Qi-fan, HE Zhi-quan. Review of pavement defect detection methods[J]. IEEE Access, 2020, 8: 14531-14544. doi: 10.1109/ACCESS.2020.2966881
[2]	徐志刚, 车艳丽, 李金龙, 等. 路面破损图像自动处理技术研究进展[J]交通运输工程学报, 2019, 19(1): 172-190. doi: 10.3969/j.issn.1671-1637.2019.01.017 XU Zhi-gang, CHE Yan-li, LI Jin-long, et al. Research progress on automatic image processing technology for pavement distress[J]. Journal of Traffic and Transportation Engineering, 2019, 19(1): 172-190. (in Chinese) doi: 10.3969/j.issn.1671-1637.2019.01.017
[3]	SZEGEDY C, LIU W, JIA Y Q, et al. Going deeper with convolutions[C]//IEEE. 2015 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2015: 1-9.
[4]	HE Kai-ming, ZHANG Xiang-yu, REN Shao-qing, et al. Deep residual learning for image recognition[C]//IEEE. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). New York: IEEE, 2016: 770-778.
[5]	YUSOF N A M, OSMAN M K, HUSSAIN Z, et al. Automated asphalt pavement crack detection and classification using deep convolution neural network[C]//IEEE. 9th IEEE International Conference on Control System, Computing and Engineering. New York: IEEE, 2019: 215-220.
[6]	PARK S, BANG S, KIM H, et al. Patch-based crack detection in black box images using convolutional neural networks[J]. Journal of Computing in Civil Engineering, 2019, 33(3): 04019017. doi: 10.1061/(ASCE)CP.1943-5487.0000831
[7]	章天杰, 韩海航. 基于残差神经网络的沥青路面裂缝识别分类研究[J]. 公路, 2021, 66(10): 24-29. doi: 10.3969/j.issn.1002-0268.2021.10.004 ZHANG Tian-jie, HAN Hai-hang. Research on identification and classification of asphalt pavement cracks using residual neural network[J]. Highway, 2021, 66(10): 24-29. (in Chinese) doi: 10.3969/j.issn.1002-0268.2021.10.004
[8]	IBRAGIMOV E, LEE H J, LEE J J, et al. Automated pavement distress detection using region based convolutional neural networks[J]. International Journal of Pavement Engineering, 2022, 23(6): 1981-1992. doi: 10.1080/10298436.2020.1833204
[9]	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. doi: 10.1109/TPAMI.2016.2577031
[10]	孙朝云, 裴莉莉, 李伟, 等. 基于改进Faster R-CNN的路面灌封裂缝检测方法研究[J]. 华南理工大学学报(自然科学版), 2020, 48(2): 84-93. https://www.cnki.com.cn/Article/CJFDTOTAL-HNLG202002011.htm SUN Zhao-yun, PEI Li-li, LI Wei, et al. Pavement sealed crack detection method based on improved faster R-CNN[J]. Journal of South China University of Technology (Natural Science Edition), 2020, 48(2): 84-93. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-HNLG202002011.htm
[11]	TRAN V P, TRAN T S, LEE H J, et al. One stage detector (RetinaNet)-based crack detection for asphalt pavements considering pavement distresses and surface objects[J]. Journal of Civil Structural Health Monitoring, 2021, 11(1): 205-222. doi: 10.1007/s13349-020-00447-8
[12]	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2): 318-327. doi: 10.1109/TPAMI.2018.2858826
[13]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//NIPS. 31st Conference on Neural Information Processing Systems. San Diego: NIPS, 2017: 1-15.
[14]	刘军, 王慧民, 张兴忠, 等. 基于Transformer的端到端路面裂缝检测方法[J]. 太原理工大学学报, 2022, 53(6): 1143-1151. https://www.cnki.com.cn/Article/CJFDTOTAL-TYGY202206021.htm LIU Jun, WANG Hui-min, ZHANG Xing-zhong, et al. End-to-end pavement crack detection method based on transformer[J]. Journal of Taiyuan University of Technology, 2022, 53(6): 1143-1151. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-TYGY202206021.htm
[15]	陈涵深, 姚明海, 瞿心昱. 基于U型全卷积神经网络的路面裂缝检测[J]. 光电工程, 2020, 47(12): 200036. https://www.cnki.com.cn/Article/CJFDTOTAL-GDGC202012007.htm CHEN Han-shen, YAO Ming-hai, QU Xin-yu. Pavement crack detection based on the U-shaped fully convolutional neural network[J]. Opto-Electronic Engineering, 2020, 47(12): 200036. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-GDGC202012007.htm
[16]	RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[J]. Lecture Notes in Computer Science, 2015, 9351: 234-241.
[17]	瞿中, 陈雯. 基于空洞卷积和多特征融合的混凝土路面裂缝检测[J]. 计算机科学, 2022, 49(3): 192-196. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJA202203026.htm QU Zhong, CHEN Wen. Concrete pavement crack detection based on dilated convolution and multi-features fusion[J]. Computer Science, 2022, 49(3): 192-196. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JSJA202203026.htm
[18]	陈良全, 王彩玲, 刘华军, 等. 基于连续注意力机制和卷积金字塔的路面裂缝检测[J]. 计算机系统应用, 2021, 30(8): 249-255. https://www.cnki.com.cn/Article/CJFDTOTAL-XTYY202108035.htm CHEN Liang-quan, WANG Cai-ling, LIU Hua-jun, et al. Pavement crack detection with continuous attention mechanism and convolution pyramid structure[J]. Computer Systems and Applications, 2021, 30(8): 249-255. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-XTYY202108035.htm
[19]	BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495. doi: 10.1109/TPAMI.2016.2644615
[20]	张伯树, 张志华, 张洋. 改进的HRNet应用于路面裂缝分割与检测[J]. 测绘通报, 2022(3): 83-89. https://www.cnki.com.cn/Article/CJFDTOTAL-CHTB202203016.htm ZHANG Bo-shu, ZHANG Zhi-hua, ZHANG Yang. Improved HRNet applied to segmentation and detection of pavement cracks[J]. Bulletin of Surveying and Mapping, 2022(3): 83-89. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-CHTB202203016.htm
[21]	WU Yang-xu, YANG Wan-ting, PAN Jin-xiao, et al. Asphalt pavement crack detection based on multi-scale full convolutional network[C]//IEEE. 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2017: 1495-1508.
[22]	HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//IEEE. 2017 IEEE Conference on Computer Vision and Pattern Recognition, New York: IEEE, 2017: 4700-4708.
[23]	LIU J W, YANG X, LAU S, et al. Automated pavement crack detection and segmentation based on two-step convolutional neural network[J]. Computer-Aided Civil and Infrastructure Engineering, 2020, 35(11): 1291-1305.
[24]	WANG S W, GONG Y C, XING J L, et al. RDSNet: a new deep architecture for reciprocal object detection and instance segmentation[C]//AIAA. 34th AAAI Conference on Artificial Intelligence. Reston: AIAA, 2020: 12208-12215.
[25]	LIN T Y, DOLLAR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//IEEE. 30th IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2017: 936-944.
[26]	SHI Y, CUI L M, QI Z Q, et al. Automatic road crack detection using random structured forests[J]. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(12): 3434-3445.
[27]	YANG F, ZHANG L, YU S J, et al. Feature pyramid and hierarchical boosting network for pavement crack detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 21(4): 1525-1535.
[28]	SONG Liang, WANG Xuan-cang. Faster region convolutional neural network for automated pavement distress detection[J]. Road Materials and Pavement Design, 2021, 22(1): 23-41.
[29]	SHEN Ting-sheng, NIE Ming-xin. Pavement damage detection based on cascade R-CNN[C]//ACM. Proceedings of the 4th International Conference on Computer Science and Application Engineering. New York: ACM, 2020: 1-5.
[30]	HE K M, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[C]//IEEE. 2017 IEEE International Conference on Computer Vision. New York: IEEE, 2017: 2980-2988.
[31]	ZHANG Y F, CHEN B, WANG J F, et al. APLCNet: automatic pixel-level crack detection network based on instance segmentation[J]. IEEE Access, 2020, 8: 199159-199170.
[32]	SHRIVASTAVA A, GUPTA A, GIRSHICK R. Training region-based object detectors with online hard example mining[C]//IEEE. 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2016: 761-769.

Relative Articles

Supplements(0)

Cited By

Proportional views

Proportional views

通讯作者: 陈斌, bchen63@163.com

1.
沈阳化工大学材料科学与工程学院沈阳 110142

Get Citation

PDF

XML

Article Metrics

Article views (650) PDF downloads(167)

Pavement crack detection method based on multi-scale feature enhancement

doi: 10.19818/j.cnki.1671-1637.2023.01.022

Abstract

References

Proportional views

Catalog

通讯作者: 陈斌, bchen63@163.com

Article Metrics

Proportional views

Related

Pavement crack detection method based on multi-scale feature enhancement

doi: 10.19818/j.cnki.1671-1637.2023.01.022

Abstract

References

Proportional views

Catalog

通讯作者: 陈斌, bchen63@163.com

Article Metrics

Proportional views

Related

Export File

Citation

Format

Content