Method for UAV adaptive cruising trajectory planning for high-risk road sections of forest roads
Article Text (Baidu Translation)
-
摘要: 为降低因森林遮蔽所引发的林区道路交通事故风险，提出了一种融合状态不确定性的深度强化学习无人机自适应巡航模型。在感知层面，设计了一种适用于林区公路场景的自适应无迹卡尔曼滤波(UKF)算法，来应对全球定位系统信号缺失；在决策层面，构建了基于状态不确定性感知的柔性演员-评论家(SUA-SAC)算法，将UKF输出的状态估计值及其协方差作为网络输入，使SUA-SAC算法能够学习到对状态估计具有更优鲁棒性的控制策略。结果表明：在训练效率方面，SUA-SAC算法的收敛速度相较于基线SAC算法和近端策略优化算法分别提升了约50%和60%；在多场景测试中，相较于SAC算法，SUA-SAC算法在无干扰、动态遮挡和强风干扰场景下的平均跟踪误差分别降低了67%、61%和66%；在定位信号缺失达20 s的测试中，SUA-SAC算法的跟踪误差所受影响较小。SUA-SAC算法能够提高无人机在林区复杂路况下的轨迹跟踪精度、飞行稳定性以及任务成功率，有助于提升林区公路的交通安全水平。Abstract: To mitigate forest road traffic accident risks caused by canopy obstruction, a deep reinforcement learning (DRL)-based adaptive cruising model for UAVs that accounts for state uncertainty was proposed. At the perception stage, an adaptive unscented Kalman filter (UKF) tailored to forest road scenarios was designed to address global positioning system (GPS) signal loss. At the decision-making stage, a state uncertainty-aware soft actor-critic (SUA-SAC) algorithm was developed, where UKF-derived state estimates and their covariance were used as network inputs, enabling SUA-SAC to learn control strategies that are more robust to state estimation. The results show that, in terms of training efficiency, the convergence speed of SUA-SAC is improved by approximately 50% and 60% compared with the baseline SAC algorithm and proximal policy optimization algorithm. In multi-scenario tests, compared with the SAC algorithm, SUA-SAC reduces the average tracking error by 67%, 61%, and 66% in scenarios without interference, dynamic occlusion, and strong wind interference, respectively. In tests involving positioning signal loss lasting up to 20 s, the tracking error of SUA-SAC is less affected. Overall, SUA-SAC improves UAV trajectory tracking accuracy, flight stability, and mission success rate under complex forest road conditions, contributing to the enhancement of traffic safety on forest roads.
-
Key words:
- low-altitude traffic /
- traffic safety /
- forest road /
- UAV cruising /
- SUA-SAC algorithm
-
图 4 三种场景下算法跟踪路径效果对比
Figure 4. Comparison of trajectory tracking performances of the algorithms under three scenario
图 6 GPS信号遮挡下横向与高程相对误差分析
Figure 6. Analysis of lateral and elevation relative errors under GPS signal obstruction
表 1 仿真环境关键参数
Table 1. Key parameters of the simulation environment
参数 值 参数 值 无人机质量/kg 0.47 物理仿真时间步长/s 1/240 训练总步数 8.0×106 经验回放缓冲区大小 1.0×106 采样批量大小 256 折扣因子 0.99 网络学习率 0.000 3 软更新系数 0.005 失败判定阈值/m 10.0 目标熵 -4 最小转弯半径/m 2.5 飞行空域边界/m [50, 50, 30] 最大设计平飞速度/ (m·s-1) 15 最大设计抗风能力/ (m·s-1) 12 
下载: 导出CSV
表 2 关键超参数设置
Table 2. Key hyperparameter settings
超参数 SUA-SAC算法 SAC算法 TD3算法 PPO算法 描述 学习率/10-4 3.0 3.0 3.0 1.0 控制网络权重更新幅度 折扣因子 0.99 0.99 0.99 0.99 对未来奖励的重视程度 批处理大小 256 256 256 每次梯度更新时使用的经验样本数量 经验回放池大小/106 1.0 1.0 1.0 存储历史经验以供重复学习的内存容量 目标熵 -4 -4 自动温度调整机制所要维持的策略熵 目标网络软更新系数 0.005 0.005 0.005 目标网络向网络软更新的混合比例 更新前步数 2 048 策略更新前，每个环境收集的经验步数 广义优势估计因子 0.95 用于平衡方差与偏差 更新轮次 10 每次收集数据后，对数据重复学习的次数 PPO裁剪系数 0.2 PPO算法中限制策略更新幅度的裁剪系数
下载: 导出CSV
表 3 各算法在不同场景的性能指标
Table 3. Performance metrics of different algorithms in various scenarios
性能指标 场景1 场景2 场景3 PPO SAC TD3 SUA-SAC PPO SAC TD3 SUA-SAC PPO SAC TD3 SUA-SAC 任务成功率/% 98.1 99.1 97.9 99.7 95.2 98.6 94.8 99.8 94.5 94.8 93.2 98.2 平均跟踪误差/m 1.76 0.85 1.82 0.28 2.31 1.15 2.85 0.45 2.95 1.48 3.42 0.51 均方根误差/m 2.05 1.02 2.25 0.34 2.85 1.38 3.45 0.53 3.51 1.75 4.05 0.62 最大误差/m 2.64 1.28 2.98 0.41 3.24 1.95 4.85 0.88 4.18 2.65 5.25 1.15 误差标准差/m 0.95 0.45 1.12 0.19 1.22 0.61 1.65 0.25 1.55 0.78 1.95 0.31 路径长度比 1.012 1.005 1.015 1.001 1.021 1.009 1.032 1.004 1.028 1.013 1.048 1.006 累计绝对急动度 18.2 9.8 19.7 4.5 29.8 15.5 38.5 8.1 35.5 19.2 48.6 9.5 控制能耗 0.82 0.65 0.87 0.48 0.91 0.72 1.12 0.55 0.98 0.81 1.35 0.61 单步平均求解耗时/ms 0.82 0.88 0.82 1.56 0.81 0.89 0.81 1.58 0.83 0.88 0.83 1.55
下载: 导出CSV
-
[1] 杨豪, 刘李彦, 张军辉, 等. 环境适应性优化的轻量化多尺度道路裂缝检测[J]. 中国公路学报, 2025, 38(7): 118-134.YANG Hao, LIU Li-yan, ZHANG Jun-hui, et al. Environmental adaptability optimization lightweight multi-scale road crack detection[J]. China Journal of Highway and Transport, 2025, 38(7): 118-134. [2] SILVA L A, LEITHARDT V R Q, BATISTA V F L, et al. Automated road damage detection using UAV images and deep learning techniques[J]. IEEE Access, 2023, 11: 62918-62931. doi: 10.1109/ACCESS.2023.3287770 [3] TUTSOY O, ASADI D, AHMADI K, et al. Minimum distance and minimum time optimal path planning with bioinspired machine learning algorithms for faulty unmanned air vehicles[J]. IEEE Transactions on Intelligent Transportation Systems, 2024, 25(8): 9069-9077. doi: 10.1109/TITS.2024.3367769 [4] LOPEZ-SANCHEZ I, MORENO-VALENZUELA J. PID control of quadrotor UAVs: A survey[J]. Annual Reviews in Control, 2023, 56: 100900. doi: 10.1016/j.arcontrol.2023.100900 [5] MOHINDRU P. Review on PID, fuzzy and hybrid fuzzy PID controllers for controlling non-linear dynamic behaviour of chemical plants[J]. Artificial Intelligence Review, 2024, 57(4): 97. doi: 10.1007/s10462-024-10743-0 [6] ZHAO B, XIAN B, ZHANG Y, et al. Nonlinear robust adaptive tracking control of a quadrotor UAV via immersion and invariance methodology[J]. IEEE Transactions on Industrial Electronics, 2015, 62(5): 2891-2902. doi: 10.1109/TIE.2014.2364982 [7] 申富媛, 李炜, 蒋栋年. 四旋翼无人机寿命预测和自主维护方法[J]. 吉林大学学报(工学版), 2023, 53(3): 841-852.SHEN Fu-yuan, LI Wei, JIANG Dong-nian. Life prediction and self-maintenance method of quadrotor unmanned aerial vehicle[J]. Journal of Jilin University (Engineering and Technology Edition), 2023, 53(3): 841-852. [8] 秦明星, 王忠, 李海龙, 等. 基于分布式模型预测的无人机编队避障控制[J]. 北京航空航天大学学报, 2024, 50(6): 1969-1981.QIN Ming-xing, WANG Zhong, LI Hai-long, et al. Obstacle avoidance control of UAV formation based on distributed model prediction[J]. Journal of Beijing University of Aeronautics and Astronautics, 2024, 50(6): 1969-1981. [9] XIE R L, MENG Z J, WANG L F, et al. Unmanned aerial vehicle path planning algorithm based on deep reinforcement learning in large-scale and dynamic environments[J]. IEEE Access, 2021, 9: 24884-24900. doi: 10.1109/ACCESS.2021.3057485 [10] YIN Y F, GUO Y, SU Q R, et al. Task allocation of multiple unmanned aerial vehicles based on deep transfer reinforcement learning[J]. Drones, 2022, 6(8): 215. doi: 10.3390/drones6080215 [11] ZUO Z Y, LIU C J, HAN Q L, et al. Unmanned aerial vehicles: Control methods and future challenges[J]. IEEE/CAA Journal of Automatica Sinica, 2022, 9(4): 601-614. doi: 10.1109/JAS.2022.105410 [12] BOUHAMED O, GHAZZAI H, BESBES H, et al. Autonomous UAV navigation: A DDPG-based deep reinforcement learning approach[C]//IEEE. 2020 IEEE International Symposium on Circuits and Systems (ISCAS). New York: IEEE, 2020: 1-5. [13] 郭靖, 鲜勇, 任乐亮, 等. 自适应斥力势场的无人机航迹规划[J/OL]. 北京航空航天大学学报, 2024, https://doi.org/10.13700/j.bh.1001-5965.2024.0569 .GUO Jing, XIAN Yong, REN Le-liang, et al. Adaptive Repulsive Potential Field for UAV Trajectory Planning[J/OL]. Journal of Beijing University of Aeronautics and Astronautics, 2024,https://doi.org/10.13700/j.bh.1001-5965.2024.0569 .[14] ZHAN G, ZHANG X M, LI Z C, et al. Multiple-UAV reinforcement learning algorithm based on improved PPO in ray framework[J]. Drones, 2022, 6(7): 166. doi: 10.3390/drones6070166 [15] 陈运翔, 苟明, 张建平, 等. 基于多智能体近端策略优化的低空异构飞行器实时三维冲突解脱方法[J]. 交通运输工程学报, 2026, 26(3): 185-197. doi: 10.19818/j.cnki.1671-1637.2026.092CHEN Yun-xiang, GOU Ming, ZHANG Jian-ping, et al. Real-time 3D conflict resolution method for low-altitude heterogeneous aircraft based on multi-agent proximal policy optimization[J]. Journal of Traffic and Transportation Engineering, 2026, 26(3): 185-197. doi: 10.19818/j.cnki.1671-1637.2026.092 [16] MA B D, LIU Z B, DANG Q Q, et al. Deep reinforcement learning of UAV tracking control under wind disturbances environments[J]. IEEE Transactions on Instrumentation and Measurement, 2023, 72: 1-13. [17] USMAN M, IRSHAD A, ASHRAF CHAUDHRY S. SAC-DeV: Secure access control for drone-assisted edge computing in e-VANETs[J]. IEEE Internet of Things Journal, 2025, 12(24): 53121-53131. doi: 10.1109/JIOT.2025.3615508 [18] 寇凯, 杨刚, 张文启, 等. 基于SAC的无人机自主导航方法研究[J]. 西北工业大学学报, 2024, 42(2): 310-318.KOU Kai, YANG Gang, ZHANG Wen-qi, et al. Exploring UAV autonomous navigation algorithm based on soft actor-critic[J]. Journal of Northwestern Polytechnical University, 2024, 42(2): 310-318. [19] 雷耀麟, 丁文锐, 罗祎喆, 等. 无人机数据采集任务中的航迹规划与资源分配优化[J]. 北京航空航天大学学报, 2025, 51(10): 3460-3470.LEI Yao-lin, DING Wen-rui, LUO Yi-zhe, et al. Trajectory planning and resource allocation optimization in UAV data collection missions[J]. Journal of Beijing University of Aeronautics and Astronautics, 2025, 51(10): 3460-3470. [20] 赵军, 何家政, 孙冰寒, 等. 基于深度强化学习的四旋翼无人机姿态控制[J]. 中国惯性技术学报, 2025, 33(3): 284-292, 300.ZHAO Jun, HE Jia-zheng, SUN Bing-han, et al. Attitude control of quadrotor unmanned aerial vehicle based on deep reinforcement learning[J]. Journal of Chinese Inertial Technology, 2025, 33(3): 284-292, 300. [21] MITTAL V, MAGHSUDI S, HOSSAIN E. Distributed cooperation under uncertainty in drone-based wireless networks: A Bayesian coalitional game[J]. IEEE Transactions on Mobile Computing, 2023, 22(1): 206-221. doi: 10.1109/TMC.2021.3073772 [22] 黄鹤, 张科, 陈永安, 等. 一种无人机航拍目标的长期跟踪算法[J]. 哈尔滨工业大学学报, 2022, 54(5): 104-116.HUANG He, ZHANG Ke, CHEN Yong-an, et al. A long-term tracking algorithm for UAV aerial photography[J]. Journal of Harbin Institute of Technology, 2022, 54(5): 104-116. [23] 张宏展, 韩鹏, 赵可, 等. 融合金字塔栅格与测向交叉的低空无人机定位[J]. 交通信息与安全, 2025, 43(3): 162-170.ZHANG Hong-zhan, HAN Peng, ZHAO Ke, et al. Low-altitude UAV positioning fusing pyramid grid and direction-finding cross-location[J]. Journal of Transport Information and Safety, 2025, 43(3): 162-170. [24] 侯佳林, 侯榕榕, 鲍跃全. 基于三维高斯溅射的桥梁自动化三维重建方法[J]. 中国公路学报, 2026, 39(1): 87-98.HOU Jia-lin, HOU Rong-rong, BAO Yue-quan. Automated 3D reconstruction of bridges based on 3D Gaussian splatting[J]. China Journal of Highway and Transport, 2026, 39(1): 87-98. [25] 李书恒, 何德峰, 廖飞, 等. 基于非线性前馈补偿的六旋翼无人机姿态稳定反步控制[J]. 上海交通大学学报, 2025, 59(12): 1891-1900.LI Shu-heng, HE De-feng, LIAO Fei, et al. Backstepping control for attitude stabilization of hexacopter based on nonlinear feedforward compensation[J]. Journal of Shanghai Jiao Tong University, 2025, 59(12): 1891-1900. [26] 房杰, 谢富源, 曹越, 等. 一种四旋翼无人机的三维轨迹规划与抗干扰轨迹跟踪控制方法[J]. 南京理工大学学报, 2025, 49(4): 494-503.FANG Jie, XIE Fu-yuan, CAO Yue, et al. A three-dimensional trajectory planning and anti-disturbance trajectory tracking control method of quadcopters[J]. Journal of Nanjing University of Science and Technology, 2025, 49(4): 494-503. [27] 蔡志浩, 陈文军, 赵江, 等. 基于动态视觉传感器的无人机目标检测与避障[J]. 北京航空航天大学学报, 2024, 50(1): 144-153.CAI Zhi-hao, CHEN Wen-jun, ZHAO Jiang, et al. Object detection and obstacle avoidance based on dynamic vision sensor for UAV[J]. Journal of Beijing University of Aeronautics and Astronautics, 2024, 50(1): 144-153. [28] 羊钊, 齐洪彪, 于阳阳, 等. 风险规避与组合策略融合的多无人机协同路径规划方法[J]. 交通运输工程学报, 2026, 26(3): 140-158. doi: 10.19818/j.cnki.1671-1637.2026.089YANG Zhao, QI Hong-biao, YU Yang-yang, et al. Integrated risk avoidance and hybrid strategy for multi-UAV cooperative path planning[J]. Journal of Traffic and Transportation Engineering, 2026, 26(3): 140-158. doi: 10.19818/j.cnki.1671-1637.2026.089 -
图(6) / 表(3)
计量
- 文章访问数: 101
- HTML全文浏览量: 33
- PDF下载量: 10
- 被引次数: 0
