Extraction and quantification of pavement alligator crack morphology based on VGG16-UNet semantic segmentation model

ZHANG Wei-guang; ZHONG Jing-tao; HUYAN Ju; MA Tao; ZHU Jun-qing; HE Liang

doi:10.19818/j.cnki.1671-1637.2023.02.012

Volume 23 Issue 2

Apr. 2023

Turn off MathJax

Article Contents

Article Navigation > Journal of Traffic and Transportation Engineering > 2023 > 23(2): 166-182

Next Previous

ZHANG Wei-guang, ZHONG Jing-tao, HUYAN Ju, MA Tao, ZHU Jun-qing, HE Liang. Extraction and quantification of pavement alligator crack morphology based on VGG16-UNet semantic segmentation model[J]. Journal of Traffic and Transportation Engineering, 2023, 23(2): 166-182. doi: 10.19818/j.cnki.1671-1637.2023.02.012

Citation:

ZHANG Wei-guang, ZHONG Jing-tao, HUYAN Ju, MA Tao, ZHU Jun-qing, HE Liang. Extraction and quantification of pavement alligator crack morphology based on VGG16-UNet semantic segmentation model[J]. Journal of Traffic and Transportation Engineering, 2023, 23(2): 166-182. doi: 10.19818/j.cnki.1671-1637.2023.02.012

Citation:

ZHANG Wei-guang, ZHONG Jing-tao, HUYAN Ju, MA Tao, ZHU Jun-qing, HE Liang. Extraction and quantification of pavement alligator crack morphology based on VGG16-UNet semantic segmentation model[J]. Journal of Traffic and Transportation Engineering, 2023, 23(2): 166-182. doi: 10.19818/j.cnki.1671-1637.2023.02.012

PDF( 46301 KB)

Extraction and quantification of pavement alligator crack morphology based on VGG16-UNet semantic segmentation model

doi: 10.19818/j.cnki.1671-1637.2023.02.012

1.
School of Transportation, Southeast University, Nanjing 210096, Jiangsu, China
2.
School of Civil Engineering, Chongqing Jiaotong University, Chongqing 400074, China

Funds:

National Key Research and Development Program of China 2020YFB1600102

National Natural Science Foundation of China 52278443

More Information

Author Bio:
ZHANG Wei-guang(1986-), male, associate professor, PhD, wgzhang@seu.edu.cn

MA Tao(1981-), male, professor, PhD, matao@seu.edu.cn
Received Date: 2022-11-04
Available Online: 2023-05-09
Publish Date: 2023-04-25

Abstract

Abstract

A pavement alligator crack segmentation technology was proposed based on the VGG16-UNet semantic segmentation model, and a quantitative evaluation method was developed for extracting alligator crack morphology via the proposed model. Alligator crack images were collected by a mobile camera under the conditions of dry pavement, waterlogged pavement, and road marking. A multi-scene alligator crack image dataset was established including light, medium, and heavy damages. The alligator crack image segmentation effects and corresponding segmentation indexes of VGG16-UNet, VGG19-UNet, PSPNet, SegNet, and DeepLab v3 + were analyzed to select the optimal segmentation model, and the alligator crack morphology features were extracted. Three contour fitting boundary methods of the circumscribed regular rectangle, minimum circumscribed rectangle, and convex hull were compared. The optimal fitting boundary was determined, and the area of alligator cracks was calculated. A calculation method of alligator crack fragmentation based on the coordinates of the extreme points of the contour boundary was proposed. The pavement alligator crack area and fragmentation features were analyzed accordingly. Research results show that compared with four semantic segmentation models of VGG19-UNet, PSPNet, SegNet, and DeepLab v3 +, VGG16-UNet has the smallest model parameter quantity of 14 710 464, and the training speed can reach 118 s for each training epoch. It possesses the highest prediction speed of 0.021 s for each image. Additionally, VGG16-UNet shows a precision of 81%, recall of 82%, harmonic mean of 0.81, and mean intersection over union (MIoU) of 0.73 on open-source Crack500 dataset, CrackTree dataset, and EdmCrack600 dataset. It outperforms the other four models and shows higher generalization ability. The proposed convex hull boundary fitting method gets the smallest alligator crack area. Compared with the boundary fitting methods of circumscribed regular rectangle and minimum circumscribed rectangle, the proposed method increases the fitting rate by 14.47% and 9.30%, respectively, and obtains the optimal pavement alligator crack contour boundary. The alligator crack fragmentation values calculated by the extreme points of the contour boundary can evaluate the damage level of alligator cracks, which solves the difficulty of existing pavement alligator crack fragmentation calculation.
- pavement engineering,
- semantic segmentation,
- alligator crack boundary fitting,
- alligator crack morphology,
- alligator crack area,
- alligator crack fragmentation

FullText(HTML)

References(50)

References

[1]	MAEDA H, SEKIMOTO Y, SETO T, et al. Road damage detection and classification using deep neural networks with smartphone images[J]. Computer-Aided Civil and Infrastructure Engineering, 2018, 33(12): 1127-1141. doi: 10.1111/mice.12387
[2]	LI B X, WANG K C P, ZHANG A, et al. Automatic classification of pavement crack using deep convolutional neural network[J]. International Journal of Pavement Engineering, 2020, 21(4): 457-463. doi: 10.1080/10298436.2018.1485917
[3]	ZHANG A, WANG K C P, LI B X, et al. Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network[J]. Computer-Aided Civil and Infrastructure Engineering, 2017, 32(10): 805-819. doi: 10.1111/mice.12297
[4]	TANG Fan-long, MA Tao, ZHANG Jun-hui, et al. Integrating three-dimensional road design and pavement structure analysis based on BIM[J]. Automation in Construction, 2020, 113: 103152. doi: 10.1016/j.autcon.2020.103152
[5]	TANG Fan-long, MA Tao, GUAN Yong-sheng, et al. Parametric modeling and structure verification of asphalt pavement based on BIM-ABAQUS[J]. Automation in Construction, 2020, 111: 103066. doi: 10.1016/j.autcon.2019.103066
[6]	CAO Jia-yu. Research on the technology of automatic pavement crack detection based on digital image processing[D]. Xi'an: Chang'an University, 2014. (in Chinese)
[7]	PACHECO-CHERREZ J, CARDENAS D, PROBST O. Measuring crack-type damage features in thin-walled composite beams using denoising and a 2D continuous wavelet transform of mode shapes[J]. Journal of Applied and Computational Mechanics, 2021, 7(1): 355-371.
[8]	LI Dong-ming, GAI Meng-ye, LI Chao-ran, et al. Research on adaptive optics image denoising algorithm based on the wavelet-based contourlet transform[J]. Laser and Optoelectronics Progress, 2015, 52(11): 97-102. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JGDJ201511013.htm
[9]	WANG Shi-fang, CHE Yan-li, LI Nan, et al. Asphalt pavement crack detection algorithm based on multi scale ridges[J]. China Journal of Highway and Transport, 2017, 30(4): 32-41. (in Chinese) doi: 10.3969/j.issn.1001-7372.2017.04.005
[10]	MA Wen-tao, FAN Chun-ling. Recognition and measurement of pavement cracks based on computer vision[J]. Electronic Measurement Technology, 2020, 43(19): 85-88. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-DZCL202019018.htm
[11]	PENG Bo, JIANG Yang-sheng, HAN Shi-fan, et al. A review of automatic pavement crack image recognition algorithms[J]. Journal of Highway and Transportation Research and Development, 2014, 31(7): 19-25. (in Chinese) doi: 10.3969/j.issn.1002-0268.2014.07.004
[12]	ZHANG Hong. Research on pavement crack detection system based on image processing[D]. Shenyang: Shenyang Aerospace University, 2018. (in Chinese)
[13]	DU Zhen-yu, YUAN Jie, XIAO Fei-peng, et al. Application of image technology on pavement distress detection: a review[J]. Measurement, 2021, 184: 109900. doi: 10.1016/j.measurement.2021.109900
[14]	MEI Qi-pei, GÜL M, AZIM M. R. Densely connected deep neural network considering connectivity of pixels for automatic crack detection[J]. Automation in Construction, 2020, 110: 103018. doi: 10.1016/j.autcon.2019.103018
[15]	ZHANG Wei-guang, ZHONG Jing-tao, YU Jian-xin, et al. Research on pavement crack detection technology based on convolution neural network[J]. Journal of Central South University (Science and Technology), 2021, 52(7): 2402-2415. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZNGD202107028.htm
[16]	ZHU Jun-qing, ZHONG Jing-tao, MA Tao, et al. Pavement distress detection using convolutional neural networks with images captured via UAV[J]. Automation in Construction, 2022, 133: 103991. doi: 10.1016/j.autcon.2021.103991
[17]	CHA Y J, CHOI W, BÜYÜKÖZTÜRK O. Deep learning- based crack damage detection using convolutional neural networks[J]. Computer-Aided Civil and Infrastructure Engineering, 2017, 32(5): 361-378. doi: 10.1111/mice.12263
[18]	DU Yu-chuan, PAN Ning, XU Zi-hao, et al. Pavement distress detection and classification based on YOLO network[J]. International Journal of Pavement Engineering, 2021, 22(13): 1659-1672. doi: 10.1080/10298436.2020.1714047
[19]	YANG Xin-cong, LI Heng, YU Yan-tao, et al. Automatic pixel-level crack detection and measurement using fully convolutional network[J]. Computer-Aided Civil and Infrastructure Engineering, 2018, 33(12): 1090-1109. doi: 10.1111/mice.12412
[20]	SHELHAMER E, LONG J, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651. doi: 10.1109/TPAMI.2016.2572683
[21]	RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//S pringer. Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin: Springer, 2015: 234-241.
[22]	BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495. doi: 10.1109/TPAMI.2016.2644615
[23]	CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834-848. doi: 10.1109/TPAMI.2017.2699184
[24]	ZOU Qin, ZHANG Zheng, LI Qing-quan, et al. DeepCrack: Learning hierarchical convolutional features for crack detection[J]. IEEE Transactions on Image Processing, 2019, 28(3): 1498-1512. doi: 10.1109/TIP.2018.2878966
[25]	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90. doi: 10.1145/3065386
[26]	SZEGEDY C, LIU W, JIA Y Q, et al. Going deeper with convolutions[C]//IEEE. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2015: 1-9.
[27]	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]//ICLR. 3rd International Conference on Learning Representations. Washington DC: ICLR, 2015: 1-14.
[28]	HE Kai-ming, ZHANG Xiang-yu, REN Shao-qing, et al. Deep residual learning for image recognition[C]//IEEE. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2016: 770-778.
[29]	SHI Ji-yuan, DANG Ji, CUI Mi-da, et al. Improvement of damage segmentation based on pixel-level data balance using VGG-UNet[J]. Applied Sciences, 2021, 11: 518. doi: 10.3390/app11020518
[30]	LIU Z, SONG Y Q, SHENG V S, et al. Liver CT sequence segmentation based with improved U-Net and graph cut[J]. Expert Systems with Applications, 2019, 126: 54-63. doi: 10.1016/j.eswa.2019.01.055
[31]	LIAN Sheng, LUO Zhi-ming, ZHONG Zhun, et al. Attention guided U-Net for accurate iris segmentation[J]. Journal of Visual Communication and Image Representation, 2018, 56: 296-304. doi: 10.1016/j.jvcir.2018.10.001
[32]	LI Gang, MA Biao, HE Shuan-hai, et al. Automatic tunnel crack detection based on U-Net and a convolutional neural network with alternately updated clique[J]. Sensors, 2020, 20(3): 717. doi: 10.3390/s20030717
[33]	ZHU Su-ya, DU Jian-chao, LI Yun-song, et al. Method for bridge crack detection based on the U-Net convolutional networks[J]. Journal of Xidian University, 2019, 46(4): 35-42. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-XDKD201904006.htm
[34]	WANG Hai, CAI Bai-xiang, CAI Ying-feng, et al. Detection of water-covered and wet areas on road pavement based on semantic segmentation network[J]. Automotive Engineering, 2021, 43(4): 485-491. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-QCGC202104007.htm
[35]	LIU Zheng-qing, CAO Yi-wen, WANG Yi-ze, et al. Computer vision-based concrete crack detection using U-net fully convolutional networks[J]. Automation in Construction, 2019, 104: 129-139. doi: 10.1016/j.autcon.2019.04.005
[36]	HUYAN Ju, LI Wei, TIGHE S, et al. CrackU-net: a novel deep convolutional neural network for pixelwise pavement crack detection[J]. Structural Control and Health Monitoring, 2020, 27: e2551.
[37]	SHA Ai-min, TONG Zheng, GAO Jie. Recognition and measurement of pavement disasters based on convolutional neural networks[J]. China Journal of Highway and Transport, 2018, 31(1): 1-10. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL201801002.htm
[38]	LI Hai-dong. Research on pavement distresses location and measurement under the heavy traffic using deep learning and UAV[D]. Xi'an: Chang'an University, 2019. (in Chinese)
[39]	YAN Ban-fu, XU Guan-ya, LUAN Jian, et al. Pavement distress detection based on Faster R-CNN and morphological operations[J]. China Journal of Highway and Transport, 2021, 34(9): 181-193. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL202109016.htm
[40]	LIU Jing-wei, YANG Xu, LAU S, et al. Automated pavement crack detection and segmentation based on two-step convolutional neural network[J]. Computer-Aided Civil and Infrastructure Engineering, 2020, 35(11): 1291-1305.
[41]	SUN Zhao-yun, MA Zhi-dan, LI Wei, et al. Pavement crack identification method based on deep convolutional neural network fusion model[J]. Journal of Chang'an University (Natural Science Edition), 2020, 40(4): 1-13. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-XAGL202004002.htm
[42]	WANG Jian-ming, LI Xiang-feng, YE Lei, et al. Medical image deblur using generative adversarial networks with channel attention[J]. Computer Science, 2021, 48(S1): 101-106. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JSJA2021S1018.htm
[43]	ZHAO Heng-shuang, SHI Jian-ping, QI Xiao-juan, et al. Pyramid scene parsing network[C]//IEEE. Proceedings of 30th IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2017: 6230-6239.
[44]	CHEN L C, ZHU Y C, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Springer. 15th European Conference on Computer Vision (ECCV). Berlin: Springer, 2018: 833-851.
[45]	YANG Fan, ZHANG Lei, YU Si-jia, et al. Feature pyramid and hierarchical boosting network for pavement crack detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 21(4): 1525-1535.
[46]	ZOU Qin, CAO Yu, LI Qing-zhou, et al. CrackTree: automatic crack detection from pavement images[J]. Pattern Recognition Letters, 2012, 33(3): 227-238.
[47]	MEI Qi-pei, GÜL M. A cost effective solution for pavement crack inspection using cameras and deep neural networks[J]. Construction and Building Materials, 2020, 256: 119397.
[48]	PENG Bo, JIANG Yang-sheng, PU Yun. Automated classification algorithm of pavement crack based on digital image processing[J]. China Journal of Highway and Transport, 2014, 27(9): 10-18, 24. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL201409003.htm
[49]	WU Wen-zhou, LI Li-fan, WANG Jie-chen. An improved Graham algorithm for determining the convex hull of planar points set[J]. Science of Surveying and Mapping, 2010, 35(6): 123-125. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-CHKD201006044.htm
[50]	WANG Shi-bing. Research and development of key technologies for pavement crack detection system[D]. Jinan: University of Jinan, 2019. (in Chinese)

Relative Articles

Supplements(0)

Cited By

Get Citation

PDF

XML

Article Metrics

Article views (1162) PDF downloads(175)

Extraction and quantification of pavement alligator crack morphology based on VGG16-UNet semantic segmentation model

doi: 10.19818/j.cnki.1671-1637.2023.02.012

Abstract

References

Catalog

Article Metrics

Related

Extraction and quantification of pavement alligator crack morphology based on VGG16-UNet semantic segmentation model

doi: 10.19818/j.cnki.1671-1637.2023.02.012

Abstract

References

Catalog

Article Metrics

Related

Export File

Citation

Format

Content