Optimization of testing and evaluation methods and recommendations for decision-making and planning technologies of autonomous driving driven by competition data

An autonomous driving algorithm competition was carried out, and an evaluation system for autonomous driving algorithms was established. The strengths and weaknesses of different types of algorithms in different scenarios were identified, and an "evaluation-diagnosis-optimization" closed-loop feedback system was formed. The defects of autonomous driving algorithms overly customized for specific scenarios were revealed. The results indicate that the highest score in the first track of the second competition is 72.79; the lowest score is 5.83, and the standard deviation of scores is 18.19. These results indicate that the test questions can distinguish the performance of different algorithms. Based on the deep mining of 582 versions of algorithm submissions, it is identified that Chinese autonomous driving algorithms perform relatively poorly in motor and non-motor mixed traffic scenarios, and the ability to handle the uncertainty of non-motorized behavior still has room for improvement. By comparing different algorithms, it is found that moderate customization for key scenarios is beneficial to improving generalization and stability of algorithms. By comparing pure rule-based architectures and pure learning-based architectures with rule-learning hybrid architectures, it is found that the rule-learning hybrid architecture achieves a better balance between safety and efficiency. Through the test analysis of segmented end-to-end architectures, it is found that such architectures need to incorporate physical constraints and safety models to break through the bottleneck of safety problems. The constructed "evaluation-diagnosis-optimization" closed-loop system provides three core implications for the research and development of decision-making and planning technologies of autonomous driving in China: establishing a benchmark scenario library to solve the "algorithm mismatch" problem, developing a data-rule hybrid architecture to cope with the safety/efficiency game, and formulating dynamic optimization standards for scenario classification to guide the transformation of technological paradigm.