Volume 21 Issue 5
Nov.  2021
Turn off MathJax
Article Contents
Article Navigation >  Journal of Traffic and Transportation Engineering  > 2021  >  21(5): 309-322
YANG Wei, HUANG Li-hong, ZHAO Xiang-mo, WANG Xiao. Puddle area segmentation of asphalt pavements based on FRRN attention and supervision[J]. Journal of Traffic and Transportation Engineering, 2021, 21(5): 309-322. doi: 10.19818/j.cnki.1671-1637.2021.05.026
Citation: YANG Wei, HUANG Li-hong, ZHAO Xiang-mo, WANG Xiao. Puddle area segmentation of asphalt pavements based on FRRN attention and supervision[J]. Journal of Traffic and Transportation Engineering, 2021, 21(5): 309-322. doi: 10.19818/j.cnki.1671-1637.2021.05.026
shu

Puddle area segmentation of asphalt pavements based on FRRN attention and supervision

doi: 10.19818/j.cnki.1671-1637.2021.05.026
  • 1.

    School of Automobile, Chang'an University, Xi'an 710064, Shaanxi, China

  • 2.

    School of Information Engineering, Chang'an University, Xi'an 710064, Shaanxi, China

  • 3.

    College of Transportation Engineering, Chang'an University, Xi'an 710064, Shaanxi, China

Funds:

National Key Research and Development Program of China 2018YFC0807502

Key Project of National Natural Science Foundation of China Automobile Joint Fund Project U1864204

Natural Science Foundation Youth Project of Shaanxi Province 2017JQ6045

More Information
  • Author Bio:

    YANG Wei(1985-), male, assistant professor, PhD, yw@chd.edu.cn

  • Received Date: 2021-04-30
    Available Online: 2021-11-13
  • Publish Date: 2021-10-01
    Abstract
  • To realize the automatic segmentation of puddle area of asphalt roads under various brightness levels and climate conditions, an attentional supervision mechanism was developed on the upsampling layer of existing full resolution residual network (FRRN). A puddle area attention module based on the full-resolution residual network (PAAM-FRRN) model was established by adding the puddle area attention module (PAAM) and the deep supervision module. An encoder-decoder structure was constructed by max-pooling and upsampling to capture the visual characteristics of puddle in a global scale. An attentional supervision mechanism was introduced in the upsampling layer to conduct the upsampling for the puddle area and fuse the features of different network layers to minimize the loss function of each network layer and optimize the overall final loss of the network. A total of 1 770 images (750 under low light, 740 under strong light and 280 under rainy weather) of asphalt pavement with puddle were collected for the five-fold cross-validation test, and the segmentation results of the puddle area were obtained. Manual annotation results were used as truth values, and the dice similarity coefficient (DSC), Jaccard similarity coefficient (JSC), precision, recall, and Hausdorff distance (HD95) were used as quantitative evaluation indexes. The segmentation effect of the proposed model was compared with the FRRN and seven other representative traditional models. Research results show that the mean values of DSC, JSC, precision, and recall obtained by the proposed model are 0.91, 0.86, 0.92, and 0.93, respectively, which are 3.41%, 6.17%, 2.22%, and 4.49% higher than those obtained by the FRRN, and are all higher than those obtained by the seven traditional comparison models. Standard deviation values of DSC, JSC, precision, and recall as computed by the proposed model are 0.12, 0.15, 0.11, and 0.12, respectively, which are 20.00%, 16.67%, 21.43%, and 25.00% lower than those of FRRN, and are all lower than those of the seven traditional comparison models. The HD95 of the proposed model is 38.56, which is lower than those of the other comparison models. Therefore, the proposed model can achieve accurate and effective segmentation of puddle areas of asphalt pavements under different brightness levels and climate conditions. 6 tabs, 11 figs, 40 refs.

     

    • FullText(HTML)
  • loading
    • References(40)
  • [1]
    季天剑, 黄晓明, 刘清泉. 部分滑水对路面附着系数的影响[J]. 交通运输工程学报, 2003, 3(4): 10-12. http://jtysgcxb.xml-journal.net/article/id/200304003

    JI Tian-jian, HUANG Xiao-ming, LIU Qing-quan. Part hydroplaning effect on pavement friction coefficient[J]. Journal of Traffic and Transportation Engineering, 2003, 3(4): 10-12. (in Chinese) http://jtysgcxb.xml-journal.net/article/id/200304003
    [2]
    周海超, 梁晨, 杨建, 等. 提升轮胎抗滑水性能的仿生方法[J]. 机械工程学报, 2015, 51(8): 125-130, 136. https://www.cnki.com.cn/Article/CJFDTOTAL-JXXB201508018.htm

    ZHOU Hai-chao, LIANG Chen, YANG Jian, et al. Bionic method for improving tire anti-hydroplaning performance[J]. Journal of Mechanical Engineering, 2015, 51(8): 125-130, 136. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JXXB201508018.htm
    [3]
    周海超, 王国林, 姜震, 等. 湿滑状态下轮胎路面摩擦特性的数值分析方法[J]. 机械工程学报, 2020, 56: 1-9. https://www.cnki.com.cn/Article/CJFDTOTAL-JXXB202021019.htm

    ZHOU Hai-chao, WANG Guo-lin, JIANG Zhen, et al. Numerical analysis method for friction characteristics of tire-pavement[J]. Journal of Mechanical Engineering, 2020, 56: 1-9. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JXXB202021019.htm
    [4]
    JONSSON P, CASSELGREN J, THORNBERG B. Road surface status classification using spectral analysis of NIR camera images[J]. IEEE Sensors Journal, 2015, 15(3): 1641-1656. doi: 10.1109/JSEN.2014.2364854
    [5]
    AMTHOR M, HARTMANN B, DENZLER J. Road condition estimation based on spatio-temporal reflection models[C]//DAGM. Proceedings of the 37th German Conference on Pattern Recognition. Aachen: DAGM, 2015: 3-15.
    [6]
    PICCARDI A, COLACE L. Optical detection of dangerous road conditions[J]. Sensors, 2019, 19(6): 1360. doi: 10.3390/s19061360
    [7]
    AKI M, ROJANAARPA T, NAKANO K, et al. Road surface recognition using laser radar for automatic platooning[J]. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(10): 2800-2810. doi: 10.1109/TITS.2016.2528892
    [8]
    NAKASHIMA S, ARAMAKI S, KITAZONO Y, et al. Application of ultrasonic sensors in road surface condition distinction methods[J]. Sensors, 2016, 16(10): 1678. doi: 10.3390/s16101678
    [9]
    KALLIRIS M, KANARACHOS S, KOTSAKIS R, et al. Machine learning algorithms for wet road surface detection using acoustic measurements[C]//IEEE. International Conference on Mechatronics. New York: IEEE, 2019: 265-270.
    [10]
    卢俊辉, 王建强, 李克强, 等. 基于路面温度和太阳辐射强度的路面状态识别方法[J]. 农业机械学报, 2010, 41(5): 21-23, 11. https://www.cnki.com.cn/Article/CJFDTOTAL-NYJX201005004.htm

    LU Juan-hui, WANG Jian-qiang, LI Ke-qiang, et al. Road condition detection based on road temperature and solar radiation[J]. Transactions of the Chinese Society of Agricultural Machinery, 2010, 41(5): 21-23, 11. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-NYJX201005004.htm
    [11]
    顾昊, 李勃, 张潇, 等. 基于偏振测量的路面积水积冰检测方法[J]. 电子测量技术, 2011, 34(7): 99-102. https://www.cnki.com.cn/Article/CJFDTOTAL-DZCL201107026.htm

    GU Hao, LI Bo, ZHANG Xiao, et al. Detection of road surface water and ice based on polarization measurement[J]. Electronic Measurement Technology, 2011, 34(7): 99-102. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-DZCL201107026.htm
    [12]
    GUERRERO-IBÁÑEZ J, ZEADALLY S, CONTRERAS-CASTILLO J, et al. Sensor technologies for intelligent transportation systems[J]. Sensors, 2018, 18(4): 1212.
    [13]
    YAMADA M, UEDA K, HORIBA I, et al. Detection of wet-road conditions from images captured by a vehicle-mounted camera[J]. Journal of Robotics and Mechatronics, 2005, 17(3): 269-276.
    [14]
    YANG H J, JANG H, JEONG D S. Detection algorithm for road surface condition using wavelet packet transform and SVM[C]//IEEE. Proceedings of the 19th Korea-Japan Joint Workshop on Frontiers of Computer Vision. New York: IEEE, 2013: 323-326.
    [15]
    KIM J, BAEK J, CHOI H, et al. Wet area and puddle detection for advanced driver assistance systems (ADAS) using a stereo camera[J]. International Journal of Control Automation and Systems, 2016, 14(1): 263-271.
    [16]
    ZHAO Jian-dong, WU Hong-qiang, CHEN Liang-liang. Road surface state recognition based on SVM optimization and image segmentation processing[J]. Journal of Advanced Transportation, 2017, 2017: 1-21.
    [17]
    刘新宇, 黄德启. 基于SVM分类器的道路湿滑图像分类方法研究[J]. 武汉理工大学学报(交通科学与工程版), 2011, 35(4): 784-787. https://www.cnki.com.cn/Article/CJFDTOTAL-JTKJ201104033.htm

    LIU Xin-yu, HUANG De-qi. Study on classifier of wet-road images based on SVM[J]. Journal of Wuhan University of Technology (Transportation Science and Engineering), 2011, 35(4): 784-787. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JTKJ201104033.htm
    [18]
    陈添丁, 王娃, 朱敬意, 等. 基于图像偏振特性的野外环境积水区域检测[J]. 光电工程, 2013, 40(4): 14-23. https://www.cnki.com.cn/Article/CJFDTOTAL-GDGC201304006.htm

    CHEN Tian-ding, WANG Wa, ZHU Jing-yi, et al. Wild environment water hazards detection based on polarization information of image[J]. Opto-Electronic Engineering, 2013, 40(4): 14-23. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-GDGC201304006.htm
    [19]
    万剑, 赵恺, 王维锋. 基于高维特征和RBP神经网络的湿滑道路图像判别方法[J]. 交通信息与安全, 2013, 31(2): 32-35. https://www.cnki.com.cn/Article/CJFDTOTAL-JTJS201302010.htm

    WAN Jian, ZHAO Kai, WANG Wei-feng. Classification of slippery road images based on high-dimensional features and RBP neural network[J]. Journal of Transport Information and Safety, 2013, 31(2): 32-35. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JTJS201302010.htm
    [20]
    LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. Nature, 2015, 521(7553): 436-444.
    [21]
    沈长青, 王旭, 王冬, 等. 基于多尺度卷积类内迁移学习的列车轴承故障诊断[J]. 交通运输工程学报, 2020, 20(5): 151-164. doi: 10.19818/j.cnki.1671-1637.2020.05.012

    SHEN Chang-qing, WANG Xu, WANG Dong, et al. Multi-scale convolutional intra-class transfer learning for train bearing fault diagnosis[J]. Journal of Traffic and Transportation Engineering, 2020, 20(5): 151-164. (in Chinese) doi: 10.19818/j.cnki.1671-1637.2020.05.012
    [22]
    ZAITOUN N M, AQEL M J. Survey on image segmentation techniques[J]. Procedia Computer Science, 2015, 65: 797-806.
    [23]
    HUANG Qing-hua, LUO Yao-zhong, ZHANG Qiang-zhi. Breast ultrasound image segmentation: a survey[J]. International Journal of Computer Assisted Radiology and Surgery, 2017, 12(3): 493-507.
    [24]
    LIU Wei-bo, WANG Zi-dong, LIU Xiao-hui, et al. A survey of deep neural network architectures and their applications[J]. Neurocomputing, 2017, 234: 11-26.
    [25]
    GARCIA-GARCIA A, ORTS-ESCOLANO S, OPREA S, et al. A survey on deep learning techniques for image and video semantic segmentation[J]. Applied Soft Computing, 2018, 70: 41-65.
    [26]
    LI J, NGUYEN C. Realtime water-hazard detection and visualisation for autonomous navigation and advanced driving assistance[C]//IEEE. 2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct. New York: IEEE, 2019: 287-288.
    [27]
    TAN Xue-feng, WANG Man-tao, HE Lin-chao, et al. Urban road water recognition based on deep learning[C]//Springer. Frontier Computing 2019. Berlin: Springer, 2020: 1387-1395.
    [28]
    HAN Xiao-feng, NGUYEN C, YOU S, et al. Single image water hazard detection using FCN with reflection attention units[C]//Springer. European Conference on Computer Vision 2018. Berlin: Springer, 2018: 105-121.
    [29]
    WANG L, WANG H. Water hazard detection using conditional generative adversarial network with mixture reflection attention units[J]. IEEE Access, 2019, 7: 167497-167506.
    [30]
    QIAO Jian-jun, WU Xiao, HE Jun-yan, et al. SWNet: a deep learning based approach for splashed water detection on road[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 2020: 1-14.
    [31]
    白岗岗, 侯精明, 韩浩, 等. 基于深度学习的道路积水智能监测方法[J]. 水资源保护, 2020, https://kns.cnki.net/kcms/detail/32.1356.TV.20200806.1036.002.html.

    BAI Gang-gang, HOU Jing-ming, HAN Hao, et al. Intelligent monitoring method for road inundation based on deep learning[J]. Water Resources Protection, 2020, https://kns.cnki.net/kcms/detail/32.1356.TV.20200806.1036.002.html. (in Chinese)
    [32]
    王海, 蔡柏湘, 蔡英凤, 等. 基于语义分割网络的路面积水与湿滑区域检测[J]. 汽车工程, 2021, 43(4): 485-491. https://www.cnki.com.cn/Article/CJFDTOTAL-QCGC202104007.htm

    WANG Hai, CAI Bai-xiang, CAI Ying-feng, et al. Detection of water-covered and wet areas on road pavement based on semantic segmentation network[J]. Automotive Engineering, 2021, 43(4): 485-491. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-QCGC202104007.htm
    [33]
    POHLEN T, HERMANS A, MATHIAS M, et al. Full-resolution residual networks for semantic segmentation in street scenes[C]//IEEE. 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2017: 3309-3318.
    [34]
    RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//Springer. Medical Image Computing and Computer-Assisted Intervention 2015. Berlin: Springer, 2015: 234-241.
    [35]
    LEE C Y, XIE S N, GALLAGHER, P W, et al. Deeply-supervised nets[J]. ArXiv, 2014: 562-570.
    [36]
    ALOM M Z, YAKOPCIC C, HASAN, M, et al. Recurrent residual U-Net for medical image segmentation[J]. Journal of Medical Imaging, 2019, 6(1): 014006.
    [37]
    SCHLEMPER J, OKTAY O, SCHAAP M, et al. Attention gated networks: learning to leverage salient regions in medical images[J]. Medical Image Analysis, 2019, 53: 197-207.
    [38]
    BEHESHTI N, JOHNSSON L. Squeeze U-Net: a memory and energy efficient image segmentation network[C]//IEEE. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. New York: IEEE, 2020: 1495-1504.
    [39]
    NOH H, HONG S, HAN B. Learning deconvolution network for semantic segmentation[C]//IEEE. 2015 IEEE International Conference on Computer Vision. New York: IEEE, 2015: 1520-1528.
    [40]
    LI Yan-wei, CHEN Xin-ze, ZHU Zheng, et al. Attention-guided unified network for panoptic segmentation[C]//IEEE. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2019: 7019-7028.
    • Relative Articles
    • Supplements(0)
    • Cited By
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索
    Get Citation
    PDF
    XML

    Article Metrics

    Article views (832) PDF downloads(113) Cited by()
    Proportional views
    Related

    Copyright《Journal of Traffic and Transportation Engineering》编辑部陕ICP备05001904号-1

    Address ：Editorial Department of Journal of Traffic and Transportation Engineering, Chang 'an University, Middle Section of South Second Ring Road, Xi 'an, Shaanxi（710064） Tel：029-82334388 Email：jygc@chd.edu.cn

    All visit:Today's visit:

    Supported by: Beijing Renhe Information Technology Co. Ltd    

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return