Volume 26 Issue 3
Mar.  2026
Turn off MathJax
Article Contents
Article Navigation >  Journal of Traffic and Transportation Engineering  > 2026  >  26(3): 185-197
Next Previous
CHEN Yun-xiang, GOU Ming, ZHANG Jian-ping, LU Wei-ning, TANG Kai, ZHANG Guang-yuan. Real-time 3D conflict resolution method for low-altitude heterogeneous aircraft based on multi-agent proximal policy optimization[J]. Journal of Traffic and Transportation Engineering, 2026, 26(3): 185-197. doi: 10.19818/j.cnki.1671-1637.2026.092
Citation: CHEN Yun-xiang, GOU Ming, ZHANG Jian-ping, LU Wei-ning, TANG Kai, ZHANG Guang-yuan. Real-time 3D conflict resolution method for low-altitude heterogeneous aircraft based on multi-agent proximal policy optimization[J]. Journal of Traffic and Transportation Engineering, 2026, 26(3): 185-197. doi: 10.19818/j.cnki.1671-1637.2026.092
shu

Real-time 3D conflict resolution method for low-altitude heterogeneous aircraft based on multi-agent proximal policy optimization

doi: 10.19818/j.cnki.1671-1637.2026.092
  • 1.

    School of Transportation and Logistics, Southwest Jiaotong University, Chengdu 610031, Sichuan, China

  • 2.

    Intelligent Management and Control of Low-altitude Traffic Key Laboratory of Sichuan Province, Chengdu 610031, Sichuan, China

  • 3.

    College of Mechanical and Electrical Engineering, Beijing Information Science Technology University, Beijing 100192, China

  • 4.

    Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing 100084, China

Funds:

National Key R&D Program 2022YFB4300903

Civil Aviation Joint Research Fund of National Natural Science Foundation of China U2433217

National Natural Science Foundation of China 52472332

Sichuan Provincial Major Science and Technology Special Project - Tackling Key Problems Initiative 2024ZDZX0044

Natural Science Foundation of Sichuan Province 2025ZNSFSC0394

More Information
  • Corresponding author: ZHANG Jian-ping, research fellow, PhD, Email: zhangjp@swjtu.edu.cn
  • Received Date: 2025-08-31
  • Accepted Date: 2025-11-27
  • Rev Recd Date: 2025-10-14
  • Publish Date: 2026-03-28
    Abstract
  • In response to the real-time three-dimensional conflict resolution for low-altitude heterogeneous aircraft, a rapidly developing operational scenario was studied, including shared airspace operations between medium-to-large fixed-wing aircraft and light small multi-rotor unmanned aerial vehicles (UAVs). A multi-agent proximal policy optimization (MAPPO)-based method was proposed with a centralized training and decentralized execution framework. Based on the operational characteristics of the two types of aircraft, a real-time three-dimensional conflict resolution strategy was established to allow fixed-wing aircraft to maintain stable flight while multi-rotor UAVs perform avoidance maneuvers. A multi-dimensional reward function was designed, taking into account collision avoidance, mission efficiency, priority, and smoothness. A priority mechanism was introduced to ensure the mission priority of fixed-wing aircraft and encourage proactive avoidance by multi-rotor UAVs. Simulation results show that baseline tests involving 5, 10, 20, and 30 light small multi-rotor UAVs all achieve a mission success rate of over 92%, with computational overhead ranging from 0.16 to 0.36 min, average conflict resolution time between 0.28 and 1.76 s, and flight conflict proportions between 0.95% and 2.18%. Through optimization of the state space, action space, and reward function, the proposed method reduces conflict resolution time by 2.25 s and improves mission success rate by 2% compared to existing methods. A foundation is thus laid for further research on the integrated operation of low-altitude heterogeneous aircraft in wide-area scenarios.

     

    • FullText(HTML)
  • loading
    • References(30)
  • [1]
    Civil Aviation Administration of China. Guiding opinions on promoting the development of civil unmanned aircraft (Draft for Public Comment)[EB/OL]. (2019-05-14)[2024-06-06]. http://www.caac.gov.cn/HDJL/YJZJ/201905/t20190514_196175.html.
    [2]
    PONS-PRATS J, ŽIVOJINOVIĆ T, KULJANIN J. On the understanding of the current status of urban air mobility development and its future prospects: Commuting in a flying vehicle as a new paradigm [J]. Transportation Research Part E: Logistics and Transportation Review, 2022, 166: 102868. doi: 10.1016/j.tre.2022.102868
    [3]
    GARROW L A, GERMAN B J, LEONARD C E. Urban air mobility: A comprehensive review and comparative analysis with autonomous and electric ground transportation for informing future research[J]. Transportation Research Part C: Emerging Technologies, 2021, 132: 103377. doi: 10.1016/j.trc.2021.103377
    [4]
    ZHANG Hong-hai, YI Jia, LI Shan, et al. Review on research of low-altitude airspace capacity evaluation[J]. Journal of Traffic and Transportation Engineering, 2023, 23(6): 78-93. doi: 10.19818/j.cnki.1671-1637.2023.06.003
    [5]
    TANG Xin-min, GU Jun-wei, ZHANG Kang, et al. Research review on the autonomous detect and avoid technologies for unmanned aerial vehicles [J]. Journal of Traffic and Transportation Engineering, 2026, 26(3): 1-24. doi: 10.19818/j.cnki.1671-1637.2026.085
    [6]
    Civil Aviation Administration of China. Notice on Issuing the Basic Classification Method of National Airspace. (2023-12-21)[2026-02-27]. http://www.caac.gov.cn/XXGK/XXGK/TZTG/202312/t20231221_222397.html.
    [7]
    LI Cheng-long, QU Wen-qiu, LI Yan-dong, et al. Overview of traffic management of urban air mobility (UAM)with eVTOL aircraft [J]. Journal of Traffic and Transportation Engineering, 2020, 20(4): 35-54. doi: 10.19818/j.cnki.1671-1637.2020.04.003
    [8]
    REICH P G. Analysis of long-range air traffic systems: Separation standards: Ⅰ [J]. Journal of Navigation, 1966, 19(1): 88-98. doi: 10.1017/S037346330004056X
    [9]
    FIORINI P, SHILLER Z. Motion planning in dynamic environments using velocity obstacles [J]. The International Journal of Robotics Research, 1998, 17(7): 760-772. doi: 10.1177/027836499801700706
    [10]
    BROOKER P. Lateral collision risk in air traffic track systems: A 'post-Reich' event model[J]. Journal of Navigation, 2003, 56(3): 399-409. doi: 10.1017/S0373463303002455
    [11]
    LIU Yang, XIANG Jin-wu, LUO Zhang-ping, et al. Short-term conflict detection algorithm for free flight in low-altitude airspace [J]. Journal of Beijing University of Aeronautics and Astronautics, 2017, 43(9): 1873-1881.
    [12]
    HERNÁNDEZ-ROMERO E, VALENZUELA A, RIVAS D. Probabilistic multi-aircraft conflict detection and resolution considering wind forecast uncertainty [J]. Aerospace Science and Technology, 2020, 105: 105973. doi: 10.1016/j.ast.2020.105973
    [13]
    GUAN Xiang-min, LYU Ren-li. Aircraft conflict resolution method based on satisfying game theory [J]. Acta Aeronautica et Astronautica Sinica, 2017, 38(S1): 120-128.
    [14]
    ZHANG Hong-hong, GAN Xu-sheng, SUN Jing-juan, et al. Analysis of low altitude UAV conflict resolution safety based on STPA-TOPAZ[J]. Acta Aeronautica et Astronautica Sinica, 2022, 43(7): 255-267.
    [15]
    ZHANG Qi-qian, WANG Zhong-ye, ZHANG Hong-hai, et al. SMILO-VTAC model based multi-aircraft conflict resolution method in complex low-altitude airspace[J]. Journal of Traffic and Transportation Engineering, 2019, 19(6): 125-136. doi: 10.19818/j.cnki.1671-1637.2019.06.012
    [16]
    CHEN Yun-xiang, ZHANG Jian-ping, WANG Zhi-yuan, et al. Safety separation calculation model for multi-rotor drones in low-altitude airspace based on avoidance strategy[J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(11): 349-365.
    [17]
    GU Zhi-ming, GAO Wen-ming, WEI Xiao-long, et al. Safety assessment technology of UAV conflict resolution based on the TOPAZ method[J]. Journal of Safety and Environment, 2016, 16(5): 51-56.
    [18]
    ZHU Dai-wu. Calculational methods of avoiding flight conflict in low altitude airspace[J]. Journal of Traffic and Transportation Engineering, 2005, 5(3): 73-76. https://transport.chd.edu.cn/article/id/200503016
    [19]
    PHAM H, BALASOORIYAN P, YILMAZ Y, et al. Conflict resolution for unmanned aerial vehicles using deep reinforcement learning[J]. Journal of Intelligent Robotic Systems, 2022, 95(3): 629-644.
    [20]
    LOQUERCIO A, MAQUEDA A I, DEL-BLANCO C R, et al. DroNet: Learning to fly by driving[J]. IEEE Robotics and Automation Letters, 2018, 3(2): 1088-1095. doi: 10.1109/LRA.2018.2795643
    [21]
    LIN C E, LAI Y H. UAV path prediction for CDR to manned aircraft in a confined airspace for cooperative mission[J]. International Journal of Aerospace Engineering, 2018, 2018: 8759836.
    [22]
    JILKOV V P, LEDET J H, LI X R. Multiple model method for aircraft conflict detection and resolution in intent and weather uncertainty[J]. IEEE Transactions on Aerospace and Electronic Systems, 2019, 55(2): 1004-1020. doi: 10.1109/TAES.2018.2867698
    [23]
    ZHAO X, LIU Y. Generalised single-agent reinforcement learning for multi-aircraft conflict resolution[J]. Aerospace Science and Technology, 2021, 112: 106649. doi: 10.1016/j.ast.2021.106649
    [24]
    LAI Z, ZHENG Z, QIU S, et al. Multi-agent deep deterministic policy gradient for air traffic conflict resolution[J]. Aerospace Science and Technology, 2021, 115: 106797. doi: 10.1016/j.ast.2021.106797
    [25]
    CHEN Y T, XU Y, YANG L, et al. General real-time three-dimensional multi-aircraft conflict resolution method using multi-agent reinforcement learning [J]. Transportation Research Part C: Emerging Technologies, 2023, 157: 104367. doi: 10.1016/j.trc.2023.104367
    [26]
    DONG S, LI W, LIU S, et al. Deep reinforcement learning for multi-agent conflict resolution in 3D airspace[J]. Aerospace Science and Technology, 2021, 110: 106412.
    [27]
    BRITTAIN M, WEI P. Long short-term memory network for aircraft conflict detection and resolution[J]. Journal of Guidance, Control, and Dynamics, 2021, 44(2): 330-342.
    [28]
    DALMAU R, ALLARD E. Air traffic control using message passing neural networks and multi-agent reinforcement learning[C]// SIDs. 10th SESAR Innovation Days. Brussels: SESAR, 2020: 158-167.
    [29]
    YU C, VELU A, VINITSKY E, et al. The surprising effectiveness of PPO in cooperative multi-agent reinforcement games[C]// NeurIPS. 36th Conference on Neural Information Processing Systems, San Diego: NeurIPS, 2022: 24611-24624.
    [30]
    WACHI A, SHEN X, SUI Y. A survey of constraint formulations in safe reinforcement learning[C]// IJCAI. Proceedings of the 33rd International Joint Conference on Artificial Intelligence. California: IJCAI, 2024: 8262-8271.
    • Relative Articles
    • Supplements(0)
    • Cited By

Catalog

    Get Citation
    PDF
    XML

    Article Metrics

    Article views (135) PDF downloads(30) Cited by()
    Related

    Website copyright Journal of Traffic and Transportation Engineering Editorial Department陕ICP备05001904号-1

    Address ：Editorial Department of Journal of Traffic and Transportation Engineering, Chang'an University, Middle Section of South 2nd Ring Road, Xi'an, China（710064） Tel：029-82334388 Email：jygc@chd.edu.cn

    All visit:Today's visit:

    Supported by: Beijing Renhe Information Technology Co. Ltd    

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return