Extraction and quantification of pavement alligator crack morphology based on VGG16-UNet semantic segmentation model
-
摘要: 提出了基于VGG16-UNet语义分割模型的路面龟裂分割技术，开发了该模型提取龟裂形态的量化评价方法；通过移动拍照设备采集路面干燥、浸水与存在道路标线影响等不同条件下的龟裂图像，构建了涵盖轻度、中度、重度3种破损程度的多场景龟裂图像数据集；分析了VGG16-UNet、VGG19-UNet、PSPNet、SegNet与DeepLab v3 + 语义分割模型的龟裂图像分割效果与相应分割指标，明确了最优分割模型并提取了龟裂形态特征；对比了外接正矩形、最小外接矩形和凸包3种轮廓形状拟合边界方法，确定了最优拟合边界，并据此计算了龟裂面积；提出了基于轮廓边界极点坐标的龟裂块度计算方法, 分析了路面龟裂面积和块度特征。研究结果表明：VGG16-UNet模型与VGG19-UNet、PSPNet、SegNet、DeepLab v3 + 这4种语义分割模型相比，VGG16-UNet具有数量为14 710 464的最小总参数量，模型训练速度可达每个世代训练时间为118 s，并能达到每个图像0.021 s的最高预测速度，同时在公开数据集Crack500、CrackTree与EdmCrack600中对复杂背景的裂缝分割准确率为81%，召回率为82%，调和均值为0.81，平均交并比为0.73，优于4种对比模型，模型泛化能力更高；凸包边界拟合方法计算龟裂面积最小，相较于外接正矩形与最小外接矩形边界拟合方法，拟合率分别提高了14.47%、9.30%，能得到最优的路面龟裂轮廓边界；基于轮廓边界极点计算的龟裂块度能有效评价龟裂破损程度，弥补了现有路面龟裂块度的计算难题。Abstract: A pavement alligator crack segmentation technology was proposed based on the VGG16-UNet semantic segmentation model, and a quantitative evaluation method was developed for extracting alligator crack morphology via the proposed model. Alligator crack images were collected by a mobile camera under the conditions of dry pavement, waterlogged pavement, and road marking. A multi-scene alligator crack image dataset was established including light, medium, and heavy damages. The alligator crack image segmentation effects and corresponding segmentation indexes of VGG16-UNet, VGG19-UNet, PSPNet, SegNet, and DeepLab v3 + were analyzed to select the optimal segmentation model, and the alligator crack morphology features were extracted. Three contour fitting boundary methods of the circumscribed regular rectangle, minimum circumscribed rectangle, and convex hull were compared. The optimal fitting boundary was determined, and the area of alligator cracks was calculated. A calculation method of alligator crack fragmentation based on the coordinates of the extreme points of the contour boundary was proposed. The pavement alligator crack area and fragmentation features were analyzed accordingly. Research results show that compared with four semantic segmentation models of VGG19-UNet, PSPNet, SegNet, and DeepLab v3 +, VGG16-UNet has the smallest model parameter quantity of 14 710 464, and the training speed can reach 118 s for each training epoch. It possesses the highest prediction speed of 0.021 s for each image. Additionally, VGG16-UNet shows a precision of 81%, recall of 82%, harmonic mean of 0.81, and mean intersection over union (MIoU) of 0.73 on open-source Crack500 dataset, CrackTree dataset, and EdmCrack600 dataset. It outperforms the other four models and shows higher generalization ability. The proposed convex hull boundary fitting method gets the smallest alligator crack area. Compared with the boundary fitting methods of circumscribed regular rectangle and minimum circumscribed rectangle, the proposed method increases the fitting rate by 14.47% and 9.30%, respectively, and obtains the optimal pavement alligator crack contour boundary. The alligator crack fragmentation values calculated by the extreme points of the contour boundary can evaluate the damage level of alligator cracks, which solves the difficulty of existing pavement alligator crack fragmentation calculation.
-
图 3 不同路面状况下的龟裂图像采集
Figure 3. Acquisition of alligator crack images under different pavement conditions
表 2 模型训练参数对比
Table 2. Comparison of model training parameters
模型 总参数量 权重/ MB 每个世代
训练时间/s每个图像
预测时间/sVGG16-UNet 14 710 464 95.5 118 0.021 VGG19-UNet 21 788 352 115.3 155 0.036 PSPNet 16 850 326 83.1 126 0.025 SegNet 15 273 264 96.4 120 0.028 DeepLab v3+ 18 352 048 101.2 133 0.031 
下载: 导出CSV
表 3 基于公开数据集分割结果
Table 3. Segmentation results on open-source datasets
模型 Crack500 CrackTree EdmCrack600 精确率 召回率 F1 平均交并比 精确率 召回率 F1 平均交并比 精确率 召回率 F1 平均交并比 VGG16-UNet 0.81 0.82 0.81 0.73 0.80 0.78 0.79 0.76 0.76 0.81 0.78 0.78 VGG19-UNet 0.80 0.81 0.81 0.72 0.76 0.80 0.78 0.70 0.71 0.79 0.75 0.72 PSPNet 0.76 0.59 0.66 0.56 0.61 0.56 0.58 0.50 0.51 0.56 0.53 0.61 SegNet 0.76 0.71 0.73 0.61 0.71 0.73 0.72 0.68 0.69 0.78 0.73 0.74 DeepLab v3+ 0.79 0.81 0.80 0.72 0.76 0.78 0.77 0.74 0.76 0.80 0.78 0.74
下载: 导出CSV
表 4 龟裂几何特征计算结果
Table 4. Computation results of alligator crack geometric characteristics
凸包边界 1 2 3 4 5 6 7 8 9 10 图中面积(像素×像素) 120 806.5 154 349 134 746 120 059 81 678 60 294 135 984 168 995 208 715 117 011 实际面积/m2 0.116 0 0.148 2 0.129 4 0.115 3 0.078 4 0.057 9 0.130 6 0.162 3 0.200 4 0.112 4 图中块度/m 370 270 260 381 253 170 200 493.6 210 266 实际块度/m 0.362 6 0.264 6 0.254 8 0.373 4 0.248 0 0.166 6 0.196 0 0.483 7 0.205 8 0.260 7
下载: 导出CSV
-
[1] MAEDA H, SEKIMOTO Y, SETO T, et al. Road damage detection and classification using deep neural networks with smartphone images[J]. Computer-Aided Civil and Infrastructure Engineering, 2018, 33(12): 1127-1141. doi: 10.1111/mice.12387 [2] LI B X, WANG K C P, ZHANG A, et al. Automatic classification of pavement crack using deep convolutional neural network[J]. International Journal of Pavement Engineering, 2020, 21(4): 457-463. doi: 10.1080/10298436.2018.1485917 [3] ZHANG A, WANG K C P, LI B X, et al. Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network[J]. Computer-Aided Civil and Infrastructure Engineering, 2017, 32(10): 805-819. doi: 10.1111/mice.12297 [4] TANG Fan-long, MA Tao, ZHANG Jun-hui, et al. Integrating three-dimensional road design and pavement structure analysis based on BIM[J]. Automation in Construction, 2020, 113: 103152. doi: 10.1016/j.autcon.2020.103152 [5] TANG Fan-long, MA Tao, GUAN Yong-sheng, et al. Parametric modeling and structure verification of asphalt pavement based on BIM-ABAQUS[J]. Automation in Construction, 2020, 111: 103066. doi: 10.1016/j.autcon.2019.103066 [6] 曹佳煜. 基于图像处理的路面裂缝自动检测技术研究[D]. 西安: 长安大学, 2014.CAO Jia-yu. Research on the technology of automatic pavement crack detection based on digital image processing[D]. Xi'an: Chang'an University, 2014. (in Chinese) [7] PACHECO-CHERREZ J, CARDENAS D, PROBST O. Measuring crack-type damage features in thin-walled composite beams using denoising and a 2D continuous wavelet transform of mode shapes[J]. Journal of Applied and Computational Mechanics, 2021, 7(1): 355-371. [8] 李东明, 盖梦野, 李超然, 等. 基于小波域的Contourlet变换法的自适应光学图像去噪算法研究[J]. 激光与光电子学进展, 2015, 52(11): 97-102. https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ201511013.htmLI Dong-ming, GAI Meng-ye, LI Chao-ran, et al. Research on adaptive optics image denoising algorithm based on the wavelet-based contourlet transform[J]. Laser and Optoelectronics Progress, 2015, 52(11): 97-102. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ201511013.htm [9] 王世芳, 车艳丽, 李楠, 等. 一种基于多尺度脊边缘的沥青路面裂缝检测算法[J]. 中国公路学报, 2017, 30(4): 32-41. doi: 10.3969/j.issn.1001-7372.2017.04.005WANG Shi-fang, CHE Yan-li, LI Nan, et al. Asphalt pavement crack detection algorithm based on multi scale ridges[J]. China Journal of Highway and Transport, 2017, 30(4): 32-41. (in Chinese) doi: 10.3969/j.issn.1001-7372.2017.04.005 [10] 马文涛, 樊春玲. 基于计算机视觉路面裂缝的识别与测量[J]. 电子测量技术, 2020, 43(19): 85-88. https://www.cnki.com.cn/Article/CJFDTOTAL-DZCL202019018.htmMA Wen-tao, FAN Chun-ling. Recognition and measurement of pavement cracks based on computer vision[J]. Electronic Measurement Technology, 2020, 43(19): 85-88. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-DZCL202019018.htm [11] 彭博, 蒋阳升, 韩世凡, 等. 路面裂缝图像自动识别算法综述[J]. 公路交通科技, 2014, 31(7): 19-25. doi: 10.3969/j.issn.1002-0268.2014.07.004PENG Bo, JIANG Yang-sheng, HAN Shi-fan, et al. A review of automatic pavement crack image recognition algorithms[J]. Journal of Highway and Transportation Research and Development, 2014, 31(7): 19-25. (in Chinese) doi: 10.3969/j.issn.1002-0268.2014.07.004 [12] 张红. 基于图像处理的路面裂缝检测系统研究[D]. 沈阳: 沈阳航空航天大学, 2018.ZHANG Hong. Research on pavement crack detection system based on image processing[D]. Shenyang: Shenyang Aerospace University, 2018. (in Chinese) [13] DU Zhen-yu, YUAN Jie, XIAO Fei-peng, et al. Application of image technology on pavement distress detection: a review[J]. Measurement, 2021, 184: 109900. doi: 10.1016/j.measurement.2021.109900 [14] MEI Qi-pei, GÜL M, AZIM M. R. Densely connected deep neural network considering connectivity of pixels for automatic crack detection[J]. Automation in Construction, 2020, 110: 103018. doi: 10.1016/j.autcon.2019.103018 [15] 张伟光, 钟靖涛, 于建新, 等. 基于机器学习和图像处理的路面裂缝检测技术研究[J]. 中南大学学报(自然科学版), 2021, 52(7): 2402-2415. https://www.cnki.com.cn/Article/CJFDTOTAL-ZNGD202107028.htmZHANG Wei-guang, ZHONG Jing-tao, YU Jian-xin, et al. Research on pavement crack detection technology based on convolution neural network[J]. Journal of Central South University (Science and Technology), 2021, 52(7): 2402-2415. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZNGD202107028.htm [16] ZHU Jun-qing, ZHONG Jing-tao, MA Tao, et al. Pavement distress detection using convolutional neural networks with images captured via UAV[J]. Automation in Construction, 2022, 133: 103991. doi: 10.1016/j.autcon.2021.103991 [17] CHA Y J, CHOI W, BÜYÜKÖZTÜRK O. Deep learning- based crack damage detection using convolutional neural networks[J]. Computer-Aided Civil and Infrastructure Engineering, 2017, 32(5): 361-378. doi: 10.1111/mice.12263 [18] DU Yu-chuan, PAN Ning, XU Zi-hao, et al. Pavement distress detection and classification based on YOLO network[J]. International Journal of Pavement Engineering, 2021, 22(13): 1659-1672. doi: 10.1080/10298436.2020.1714047 [19] YANG Xin-cong, LI Heng, YU Yan-tao, et al. Automatic pixel-level crack detection and measurement using fully convolutional network[J]. Computer-Aided Civil and Infrastructure Engineering, 2018, 33(12): 1090-1109. doi: 10.1111/mice.12412 [20] SHELHAMER E, LONG J, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651. doi: 10.1109/TPAMI.2016.2572683 [21] RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//S pringer. Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin: Springer, 2015: 234-241. [22] BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495. doi: 10.1109/TPAMI.2016.2644615 [23] CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834-848. doi: 10.1109/TPAMI.2017.2699184 [24] ZOU Qin, ZHANG Zheng, LI Qing-quan, et al. DeepCrack: Learning hierarchical convolutional features for crack detection[J]. IEEE Transactions on Image Processing, 2019, 28(3): 1498-1512. doi: 10.1109/TIP.2018.2878966 [25] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90. doi: 10.1145/3065386 [26] SZEGEDY C, LIU W, JIA Y Q, et al. Going deeper with convolutions[C]//IEEE. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2015: 1-9. [27] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]//ICLR. 3rd International Conference on Learning Representations. Washington DC: ICLR, 2015: 1-14. [28] HE Kai-ming, ZHANG Xiang-yu, REN Shao-qing, et al. Deep residual learning for image recognition[C]//IEEE. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2016: 770-778. [29] SHI Ji-yuan, DANG Ji, CUI Mi-da, et al. Improvement of damage segmentation based on pixel-level data balance using VGG-UNet[J]. Applied Sciences, 2021, 11: 518. doi: 10.3390/app11020518 [30] LIU Z, SONG Y Q, SHENG V S, et al. Liver CT sequence segmentation based with improved U-Net and graph cut[J]. Expert Systems with Applications, 2019, 126: 54-63. doi: 10.1016/j.eswa.2019.01.055 [31] LIAN Sheng, LUO Zhi-ming, ZHONG Zhun, et al. Attention guided U-Net for accurate iris segmentation[J]. Journal of Visual Communication and Image Representation, 2018, 56: 296-304. doi: 10.1016/j.jvcir.2018.10.001 [32] LI Gang, MA Biao, HE Shuan-hai, et al. Automatic tunnel crack detection based on U-Net and a convolutional neural network with alternately updated clique[J]. Sensors, 2020, 20(3): 717. doi: 10.3390/s20030717 [33] 朱苏雅, 杜建超, 李云松, 等. 采用U-Net卷积网络的桥梁裂缝检测方法[J]. 西安电子科技大学学报, 2019, 46(4): 35-42. https://www.cnki.com.cn/Article/CJFDTOTAL-XDKD201904006.htmZHU Su-ya, DU Jian-chao, LI Yun-song, et al. Method for bridge crack detection based on the U-Net convolutional networks[J]. Journal of Xidian University, 2019, 46(4): 35-42. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-XDKD201904006.htm [34] 王海, 蔡柏湘, 蔡英凤, 等. 基于语义分割网络的路面积水与湿滑区域检测[J]. 汽车工程, 2021, 43(4): 485-491. https://www.cnki.com.cn/Article/CJFDTOTAL-QCGC202104007.htmWANG Hai, CAI Bai-xiang, CAI Ying-feng, et al. Detection of water-covered and wet areas on road pavement based on semantic segmentation network[J]. Automotive Engineering, 2021, 43(4): 485-491. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-QCGC202104007.htm [35] LIU Zheng-qing, CAO Yi-wen, WANG Yi-ze, et al. Computer vision-based concrete crack detection using U-net fully convolutional networks[J]. Automation in Construction, 2019, 104: 129-139. doi: 10.1016/j.autcon.2019.04.005 [36] HUYAN Ju, LI Wei, TIGHE S, et al. CrackU-net: a novel deep convolutional neural network for pixelwise pavement crack detection[J]. Structural Control and Health Monitoring, 2020, 27: e2551. [37] 沙爱民, 童峥, 高杰. 基于卷积神经网络的路表病害识别与测量[J]. 中国公路学报, 2018, 31(1): 1-10. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL201801002.htmSHA Ai-min, TONG Zheng, GAO Jie. Recognition and measurement of pavement disasters based on convolutional neural networks[J]. China Journal of Highway and Transport, 2018, 31(1): 1-10. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL201801002.htm [38] 李海东. 基于深度学习和无人机的重交通荷载公路病害定位和测量研究[D]. 西安: 长安大学, 2019.LI Hai-dong. Research on pavement distresses location and measurement under the heavy traffic using deep learning and UAV[D]. Xi'an: Chang'an University, 2019. (in Chinese) [39] 晏班夫, 徐观亚, 栾健, 等. 基于Faster R-CNN与形态法的路面病害识别[J]. 中国公路学报, 2021, 34(9): 181-193. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL202109016.htmYAN Ban-fu, XU Guan-ya, LUAN Jian, et al. Pavement distress detection based on Faster R-CNN and morphological operations[J]. China Journal of Highway and Transport, 2021, 34(9): 181-193. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL202109016.htm [40] LIU Jing-wei, YANG Xu, LAU S, et al. Automated pavement crack detection and segmentation based on two-step convolutional neural network[J]. Computer-Aided Civil and Infrastructure Engineering, 2020, 35(11): 1291-1305. [41] 孙朝云, 马志丹, 李伟, 等. 基于深度卷积神经网络融合模型的路面裂缝识别方法[J]. 长安大学学报(自然科学版), 2020, 40(4): 1-13. https://www.cnki.com.cn/Article/CJFDTOTAL-XAGL202004002.htmSUN Zhao-yun, MA Zhi-dan, LI Wei, et al. Pavement crack identification method based on deep convolutional neural network fusion model[J]. Journal of Chang'an University (Natural Science Edition), 2020, 40(4): 1-13. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-XAGL202004002.htm [42] 王建明, 黎向锋, 叶磊, 等. 基于信道注意结构的生成对抗网络医学图像去模糊[J]. 计算机科学, 2021, 48(S1): 101-106. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJA2021S1018.htmWANG Jian-ming, LI Xiang-feng, YE Lei, et al. Medical image deblur using generative adversarial networks with channel attention[J]. Computer Science, 2021, 48(S1): 101-106. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JSJA2021S1018.htm [43] ZHAO Heng-shuang, SHI Jian-ping, QI Xiao-juan, et al. Pyramid scene parsing network[C]//IEEE. Proceedings of 30th IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2017: 6230-6239. [44] CHEN L C, ZHU Y C, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Springer. 15th European Conference on Computer Vision (ECCV). Berlin: Springer, 2018: 833-851. [45] YANG Fan, ZHANG Lei, YU Si-jia, et al. Feature pyramid and hierarchical boosting network for pavement crack detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 21(4): 1525-1535. [46] ZOU Qin, CAO Yu, LI Qing-zhou, et al. CrackTree: automatic crack detection from pavement images[J]. Pattern Recognition Letters, 2012, 33(3): 227-238. [47] MEI Qi-pei, GÜL M. A cost effective solution for pavement crack inspection using cameras and deep neural networks[J]. Construction and Building Materials, 2020, 256: 119397. [48] 彭博, 蒋阳升, 蒲云. 基于数字图像处理的路面裂缝自动分类算法[J]. 中国公路学报, 2014, 27(9): 10-18, 24. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL201409003.htmPENG Bo, JIANG Yang-sheng, PU Yun. Automated classification algorithm of pavement crack based on digital image processing[J]. China Journal of Highway and Transport, 2014, 27(9): 10-18, 24. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL201409003.htm [49] 吴文周, 李利番, 王结臣. 平面点集凸包Graham算法的改进[J]. 测绘科学, 2010, 35(6): 123-125. https://www.cnki.com.cn/Article/CJFDTOTAL-CHKD201006044.htmWU Wen-zhou, LI Li-fan, WANG Jie-chen. An improved Graham algorithm for determining the convex hull of planar points set[J]. Science of Surveying and Mapping, 2010, 35(6): 123-125. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-CHKD201006044.htm [50] 王世兵. 路面裂缝检测系统关键技术的研究与开发[D]. 济南: 济南大学, 2019.WANG Shi-bing. Research and development of key technologies for pavement crack detection system[D]. Jinan: University of Jinan, 2019. (in Chinese) -
点击查看大图
图(12) / 表(4)
计量
- 文章访问数: 671
- HTML全文浏览量: 197
- PDF下载量: 147
- 被引次数: 0
