Search

Scenario-based testing will help to validate automated driving systems (ADS) and establish safer road traffic. To date, most data-driven test scenario generation methods rely primarily on one data source such as police accident data (PD), naturalistic driving studies, or video-based traffic observations (VOs). However, none of these data sources perfectly satisfies all the layers of the six-layer model for the description of test scenarios. Moreover, not all available data sources cover the same location and period of time. Therefore, we fused information from 1,648 scenarios extracted from a German VO with information from 74 scenarios extracted from German PD into a comprehensive new PD* database. Finally, PD* consisted of 74 accident scenarios extended, for example, by variables containing the dynamic information of the VO scenarios. Thus, PD* contained more than 350 variables, whereas PD contained only 269 and VO only 122 variables. For fusion, we followed the Find-Unify-Synthesize-Evaluation (FUSE) for Representativity (FUSE4Rep) process model using statistical matching. Subsequently, we derived three logical scenarios from PD* to test an autonomous emergency braking system (AEB) in a stochastic traffic simulation incorporating driver-behavior models. The quality of the fusion itself was satisfactory, and we propose improving the VO data collection process and observation time to obtain even better results.

Automated driving systems should be able to avoid road traffic accidents and drive more safely than human drivers do. Test scenarios derived from real-world data such as police accident data can help assess the safety performance of automated driving systems. In many countries, the police collect data and information about nearly every accident, resulting in a representative sample. However, the collected accident data often do not contain the exact conflicts that cause an accident. Therefore, we estimated the globally known three-digit accident type for German police accident data, describing accident-causing conflicts. The data supplemented by the estimated three-digit accident type can then be used in subsequent test scenario generation. Accordingly, this study presents the first classification model for predicting 30 types of turning accidents. We tested a CatBoost model and a large-language model called BERT, using different feature sets and model designs. Overall, the CatBoost model performed best when using accident descriptions and non-text features such as collision type. Anomaly detection performed before model training revealed additional knowledge-driven miscoding in the police data collection. In conclusion, the model can predict common accident types, such as left-turns with an oncoming straight-ahead driver. In contrast, the model fails to predict rare accident types, such as left-turns (with a lit arrow sign) with oncoming traffic. Future studies should focus on optimizing the developed model, handling data imbalances, and building models using police accident data from other countries.

This survey aims to provide an overview of various methods for generating data-driven test scenarios for assessing automated driving systems (ADSs). The survey updates the overall process of scenario generation and categorizes the current methods using a systematic literature review of 64 studies identified between 2017 and 01/2023. Overall, we demonstrate that the data-driven scenario generation process should be updated by another process step, scenario fusion, leading to seven process steps: 1) scope definition, 2) primary data source selection, 3) primary data collection, 4) scenario identification, 5) scenario fusion, 6) scenario generation, and 7) scenario evaluation. “Scenario fusion” aims to fuse scenarios identified in different data sources for a better coverage of the ADSs’ operational design domains (ODDs) and a more comprehensive scenario description. Moreover, we introduce an improved definition for the representativity of test scenario catalogs, which helps improve the collection of traffic data using sampling plans. Also, we show that real driving and police accident data are the most commonly used data input sources. Besides, we illustrate that the ODD is often not defined. Finally, we discuss that the standardization of test scenario generation is difficult because most methods do not address specific ADSs and test environments, and do not provide standardized interfaces. Overall, we recommend comparing existing approaches using the same input data and researching the mutual supplementation of the existing methods. Finally, pre-defined case studies, further standardized terminology, and standards for test execution and evaluation can help speed up the standardization process.