Search

This chapter introduces a novel data synthesis framework for validation of perception functions based on machine learning to ensure the safety and functionality of these systems, specifically in the context of automated driving. The main contributions are the introduction of a generative, parametric description of three-dimensional scenarios in a validation parameter space, and layered scene generation process to reduce the computational effort. Specifically, we combine a module for probabilistic scene generation, a variation engine for scene parameters, and a more realistic sensor artifacts simulation. The work demonstrates the effectiveness of the framework for the perception of pedestrians in urban environments based on various deep neural networks (DNNs) for semantic segmentation and object detection. Our approach allows a systematic evaluation of a high number of different objects and combined with our variational approach we can effectively simulate and test a wide range of additional conditions as, e.g., various illuminations. We can demonstrate that our generative approach produces a better approximation of the spatial object distribution to real datasets, compared to hand-crafted 3D scenes.

This contribution discusses the use of variational data synthesis as a tool to analyze and understand limitations of performance of DNNs (deep neural networks) in perception tasks. To date, no universally accepted methodologies for validating ML (Machine Learning) -based perception exist. Instead of aiming for the randomized acquisition of huge amounts of validation data, either from real world capture or from simulation, we propose a guided concept to analyze perception performance using systematic parameter variations. The concept is based on parameterized, generative content used for data synthesis in our validation engine. The latter is composed of the actual data synthesis module, automated execution and evaluation of the perception function under test and a control module, which allows specification of parameter variation towards a validation goal. Further we investigate the use of physical parameters, like object occlusion rates and pixel area for the identification of critical cases for perception. We present experiments for semantic segmentation of pedestrians in an urban environment using two different DNN algorithms.