Stability model verification and control strategy optimization of train platoon based on stochastic priced timed game
-
摘要: 为保证列车队列运行安全并提高队列稳定性，研究了列车队列稳定性模型验证与控制策略优化问题；基于车-车通信的列车队列采用等空间间隔、等时间间隔和变时距3种控制策略，利用随机价格时间博弈自动机，建立了包含领航列车和跟随列车的队列控制模型，分析了模型的队列稳定性；在保证列车运行安全的前提下，以列车的相对位置差、相对速度差和时间间隔差为成本函数，通过队列随机价格时间博弈自动机模型获得控制策略集；利用Q-Learning方法得到队列的最优驾驶策略，验证队列运行的安全性和稳定性；结合列车运行追踪场景，进行队列的稳定性分析。仿真结果表明：通过形式化验证，采用3种控制策略下的队列安全性得到了保证；通过随机价格时间博弈控制、协方差优化控制和Q-Learning方法对比PID控制，等空间间隔策略下的队列稳定性误差最大值分别减小到了0.19%、0.18%和0.11%，等时间间距策略下的队列稳定性误差最大值分别减小到了30.21%、10.34%和9.24%，变时距策略下队列稳定性误差最大值分别为118.27%、56.09%和39.67%，可见，采用Q-Learning方法的随机价格时间博弈理论能在安全前提下提高列车队列稳定性。Abstract: In order to ensure the safe operation of the train platoon and improve the platoon stability, the problems of stability model verification and control strategy optimization of the train platoon were studied. The train platoon based on train-train communication adopted three control strategies including constant distance interval, constant time interval, and variable time interval. A platoon control model consisting of a leader train and following trains was established by using the stochastic priced timed game(SPTG) automata, and the stability of the platoon model was analyzed. Under the premise of ensuring the safety of train operation, the relative position difference, relative speed difference, and time interval difference of the trains were taken as the cost function, and a control strategy set was obtained by the SPTG automata model of the platoon. The optimal driving strategy of the platoon was obtained by the Q-Learning method, and the safety and stability of the platoon operation were verified. In addition, the train operation tracking scenario was utilized to analyze the stability of the platoon. Simulation results show that through formal verification, the safety of the train platoon under the three control strategies is guaranteed. For SPTG theory, covariance optimization control, and Q-Learning method compared with PID control, the maximum errors of platoon stability under the constant distance interval strategy reduce to 0.19%, 0.18% and 0.11%, respectively, the maximum errors of platoon stability under the constant time interval strategy reduce to 30.21%, 10.34%, and 9.24%, respectively, and the maximum errors of platoon stability under the variable time interval strategy are 118.37%, 56.12%, and 39.80%, respectively. Therefore, the SPTG theory using the Q-Learning method can improve the stability of the train platoon under the premise of safety.
-
图 1 车队通信拓扑结构与列车控制器
Figure 1. Train platoon communication topology structure and train controllers
图 11 等空间间隔控制策略下速度曲线
Figure 11. Speed curves based on constant distance interval control strategy
图 12 等空间间隔控制策略下稳定性曲线
Figure 12. Stability curves based on constant distance interval control strategy
图 13 等时间间隔控制策略下速度曲线
Figure 13. Speed curves based on constant time interval control strategy
图 14 等时间间隔控制策略下稳定性曲线
Figure 14. Stability curves based on constant time interval control strategy
图 15 变时距控制策略下速度曲线
Figure 15. Speed curves based on variable time interval control strategy
图 16 变时距控制策略下稳定性曲线
Figure 16. Stability curves based on variable time interval control strategy
表 1 各控制策略参数
Table 1. Parameters for each control strategy
控制策略 ji fi 固定空间间隔控制策略 $ \frac{k}{m} $ $ \frac{c}{m} $ 固定时间间隔控制策略 $ \frac{k}{m} $ $ \frac{c+k h}{m} $ 可变时距控制策略 $ \frac{k}{m} $ $ \frac{c+k \bar{v}}{m} $ 
下载: 导出CSV
表 2 PID控制参数选取
Table 2. Selection of PID control parameters
策略 列车 KP KI KD 等时间间隔控制策略 1 0.040 0.000 20 0.201 2 0.040 0.000 10 0.150 3 0.030 0.000 10 0.101 等空间间隔控制策略 1 0.045 0.000 21 0.101 2 0.040 0.000 20 0.140 3 0.025 0.000 20 0.101 变时距控制策略 1 0.004 0.000 10 0.150 2 0.003 0.001 50 0.015 3 0.030 0.005 00 0.010
下载: 导出CSV
表 3 等空间间隔控制策略稳定性分析
Table 3. Stability analysis based on constant distance interval control strategy
控制方式 稳定性误差最大值/10-3 方差 z1 z2 z1 z2 PID控制 790.00 300.00 2.14×10-1 7.10×10-2 随机运行控制 1.48 1.54 4.42×10-7 2.58×10-7 Q-Learning优化 0.90 0.59 7.60×10-8 3.00×10-9 协方差优化 1.45 1.13 1.01×10-7 4.90×10-8
下载: 导出CSV
表 4 等时间间隔控制策略稳定性分析
Table 4. Stability analysis based on constant time interval control strategy
控制方式 稳定性误差最大值/10-3 方差 z1 z2 z1 z2 分子 分母 分子 分母 PID控制 14.50 4.25 1 543.82 11 632 348.48 125.85 12 813 639.57 随机运行控制 4.38 2.88 6.94 11 777 769.37 3.97 11 779 156.60 Q-Learning优化 1.34 1.05 1.71 12 096 770.22 0.92 12 097 122.82 协方差优化 1.50 1.18 2.00 12 090 529.36 1.05 12 092 098.93
下载: 导出CSV
表 5 变时距控制策略稳定性分析
Table 5. Stability analysis based on variable time interval control strategy
控制方式 稳定性误差最大值 方差 z1 z2 z1 z2 PID控制 7.06 2.88 1.171 0.005 随机运行控制 8.35 6.12 3.629 1.944 Q-Learning优化 2.80 2.12 0.648 0.389 协方差优化 3.96 2.95 0.907 0.518
下载: 导出CSV
-
[1] 《铁道技术监督》编辑部. 新时代交通强国铁路先行规划纲要[J]. 铁道技术监督, 2020, 48(9): 1-6, 24. doi: 10.3969/j.issn.1006-9178.2020.09.001Editorial Department of Railway Quality Control. Outline of powerful nation railway advance planning in the new era[J]. Railway Quality Control, 2020, 48(9): 1-6, 24. (in Chinese) doi: 10.3969/j.issn.1006-9178.2020.09.001 [2] 张宏强. 移动闭塞条件下高速列车追踪运行控制算法研究[D]. 兰州: 兰州交通大学, 2020.ZHANG Hong-qiang. Research on control algorithm for tracking operation of high-speed trains under moving block[D]. Lanzhou: Lanzhou Jiaotong University, 2020. (in Chinese) [3] FLAMMINI F, MARRONE S, NARDONE R, et al. Towards railway virtual coupling[C]//IEEE. 2018 IEEE International Conference on Electrical Systems for Aircraft, Railway, Ship Propulsion and Road Vehicles and International Transportation Electrification Conference (ESARS-ITEC). New York: IEEE, 2018: 1-6. [4] DI MEO C, DI VAIO M, FLAMMINI F, et al. ERTMS/ETCS virtual coupling: proof of concept and numerical analysis[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 21(6): 2545-2556. doi: 10.1109/TITS.2019.2920290 [5] 崔俊锋, 刘岭. 虚拟耦合在列控系统应用的关键技术需求研究[J]. 铁路通信信号工程技术, 2019, 16(10): 1-5. doi: 10.3969/j.issn.1673-4440.2019.10.001CUI Jun-feng, LIU Ling. Research on key technology requirements of virtual coupling in application of train control system[J]. Railway Signalling and Communication Engineering, 2019, 16(10): 1-5. (in Chinese) doi: 10.3969/j.issn.1673-4440.2019.10.001 [6] LEI L, LU J H, JIANG Y M, et al. Stochastic delay analysis for train control services in next-generation high-speed railway communications system[J]. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(1): 48-64. doi: 10.1109/TITS.2015.2450751 [7] MANDUCHI G, LUCHETTA A, SOPPELSA A, et al. From distributed to multicore architecture in the RFX-mod real time control system[J]. Fusion Engineering and Design, 2014, 89(3): 224-232. doi: 10.1016/j.fusengdes.2013.07.002 [8] GURNÍK P. Next generation train control (NGTC): more effective railways through the convergence of main-line and urban train control systems[J]. Transportation Research Procedia, 2016, 14: 1855-1864. doi: 10.1016/j.trpro.2016.05.152 [9] 陈明亮, 宁滨, 荀径, 等. 列车编队运行队列稳定性分析[C]// 中国自动化学会控制理论专业委员会. 第三十八届中国控制会议论文集. 北京: 中国自动化学会控制理论专业委员会, 2019: 234-239.CHEN Ming-liang, NING Bin, XUN Jing, et al. Platoon stability analysis of train formation[C]//Technical Committee on Control Theory, Chinese Association Automation. Proceedings of the 38th Chinese Control Conference. Beijing: Technical Committee on Control Theory, Chinese Association Automation, 2019: 234-239. (in Chinese) [10] DONG H R, GAO S G, NING B. Cooperative control synthesis and stability analysis of multiple trains under moving signaling systems[J]. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(10): 2730-2738. doi: 10.1109/TITS.2016.2518649 [11] WANG M, LI H H, GAO J, et al. String stability of heterogeneous platoons with non-connected automated vehicles[C]//IEEE. 2017 IEEE 20th International Conference on Intelligent Transportation Systems. New York: IEEE, 2017: 1-8. [12] GAO S G, DONG H R, NING B, et al. Cooperative adaptive bidirectional control of a train platoon for efficient utility and string stability[J]. Chinese Physics B, 2015, 24(9): 090506. doi: 10.1088/1674-1056/24/9/090506 [13] RASHID A, SIDDIQUE U, HASAN O. Formal verification of platoon control strategies[C]//Springer. 16th International Conference on Software Engineering and Formal Methods. Berlin: Springer, 2018: 223-238. [14] 吕继东. 列车运行控制系统分层形式化建模与验证分析[D]. 北京: 北京交通大学, 2011.LYU Ji-dong. Hierarchical formal modeling and verification traincontrol system[D]. Beijing: Beijing Jiaotong University, 2011. (in Chinese) [15] 张锦坤, 杨孟飞, 乔磊, 等. 基于有限状态机的操作系统需求层形式化验证[J]. 空间控制技术与应用, 2019, 45(2): 48-55. doi: 10.3969/j.issn.1674-1579.2019.02.007ZHANG Jin-kun, YANG Meng-fei, QIAO Lei, et al. Formal verification of operating system requirements layer based on finite state machine[J]. Aerospace Control and Application, 2019, 45(2): 48-55. (in Chinese) doi: 10.3969/j.issn.1674-1579.2019.02.007 [16] BACCI G, BOUYER P, FAHRENBERG U, et al. Optimal and robust controller synthesis using energy timed automata with uncertainty[J]. Formal Aspects of Computing, 2021, 33(1): 3-25. doi: 10.1007/s00165-020-00521-4 [17] FAN C C, QIN Z Y, MATHUR U, et al. Controller synthesis for linear system with reach-avoid specifications[J]. IEEE Transactions on Automatic Control, 2022, 67(4): 1713-1727. doi: 10.1109/TAC.2021.3069723 [18] HAI Xing-shuo, WANG Zi-li, FENG Qiang, et al. A novel adaptive pigeon-inspired optimization algorithm based on evolutionary game theory[J]. Science China Information Sciences, 2021, 64(3): 1-2. [19] ABDOOS M. A cooperative multiagent system for traffic signal control using game theory and reinforcement learning[J]. IEEE Intelligent Transportation Systems Magazine, 2021, 13(4): 6-16. doi: 10.1109/MITS.2020.2990189 [20] 曹源, 唐涛, 徐田华, 等. 形式化方法在列车运行控制系统中的应用[J]. 交通运输工程学报, 2010, 10(1): 112-126. doi: 10.3969/j.issn.1671-1637.2010.01.020CAO Yuan, TANG Tao, XU Tian-hua, et al. Application of formal methods in train control system[J]. Journal of Traffic and Transportation Engineering, 2010, 10(1): 112-126. (in Chinese) doi: 10.3969/j.issn.1671-1637.2010.01.020 [21] PENG G H, SUN D H. A dynamical model of car-following with the consideration of the multiple information of preceding cars[J]. Physics Letters A, 2010, 374(15/16): 1694-1698. [22] JIANG R, HU M B, ZHANG H M, et al. On some experimental features of car-following behavior and how to model them[J]. Transportation Research Part B: Methodological, 2015, 80: 338-354. doi: 10.1016/j.trb.2015.08.003 [23] 王鹏, 李开成, 刘雨. 车车通信技术在列控系统中的应用研究[J]. 铁道通信信号, 2016, 52(7): 62-65.WANG Peng, LI Kai-cheng, LIU Yu. Application research of train-train communication technology in train control system[J]. Railway Signalling and Communication, 2016, 52(7): 62-65. (in Chinese) [24] 靳东明, 李博. 基于车-车通信的列车运行控制系统中的移动授权计算分析[J]. 电脑知识与技术, 2018, 14(1): 259-263. https://www.cnki.com.cn/Article/CJFDTOTAL-DNZS201801112.htmJIN Dong-ming, LI Bo. Analysis of movement authority calculation for train operation control system based on vehicle to vehicle communication[J]. Computer Knowledge and Technology, 2018, 14(1): 259-263. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-DNZS201801112.htm [25] 荀径, 陈明亮, 宁滨, 等. 虚拟重联条件下地铁列车追踪运行性能衡量[J]. 北京交通大学学报, 2019, 43(1): 96-103.XUN Jing, CHEN Ming-liang, NING Bin, et al. Train tracking performance measurement under virtual coupling in subway[J]. Journal of Beijing Jiaotong University, 2019, 43(1): 96-103. (in Chinese) [26] ZHU L, HE Y, YU F R, et al. Communication-based train control system performance optimization using deep reinforcement learning[J]. IEEE Transactions on Vehicular Technology, 2017, 12(66): 10705-10717. [27] GOODARZI E V, HOUSHMAND M, VALILAI O F, et al. Manufacturing cloud service composition based on the non-cooperative and cooperative game theory[C]//IEEE. 2020 IEEE International Conference on Industrial Engineering and Engineering Management. New York: IEEE, 2020: 1122-1125. [28] LI Jia-lin, DONG Ping. A dynamic adaptation mechanism of network resource based on fuzzy Q-Learning in high-speed mobile environment[C]//IEEE. Proceedings of 2020 IEEE 4th Information Technology, Networking, Electronic and Automation Control Conference. New York: IEEE, 2020: 1079-1083. [29] WANG Ye-zheng, WANG Zi-dong, ZOU Lei, et al. H∞PID control for discrete-time fuzzy systems with infinite- distributed delays under round-robin communication protocol[J]. IEEE Transactions on Fuzzy Systems, 2022, 30(6): 1875-1888. doi: 10.1109/TFUZZ.2021.3069329 [30] DAVID A, JENSEN P G, LARSEN K G, et al. On time with minimal expected cost[J]. Lecture Notes in Computer Science, 2014, 8837: 129-145. [31] 康仁伟. 基于时间自动机的CTCS-3级列控系统建模方法与验证研究[D]. 北京: 北京交通大学, 2013.KANG Ren-wei. The research on modeling and verification of Chinese train control system level 3 based on timed automata[D]. Beijing: Beijing Jiaotong University, 2013. (in Chinese) [32] ALUR R, DILL D L. A theory of timed automata[J]. Theoretical Computer Science, 1994, 126(2): 183-235. [33] DAVID A, JENSEN P G, LARSEN K G, et al. Uppaal stratego[C]//Springer. 2015 International Conference on Tools and Algorithms for the Construction and Analysis of Systems. Berlin: Springer, 2015: 206-211. -
点击查看大图
图(16) / 表(5)
计量
- 文章访问数: 274
- HTML全文浏览量: 140
- PDF下载量: 65
- 被引次数: 0
