Benchmarking of a LiDAR Sensor for use as Pseudo Ground Truth in Automotive Perception.

Abstract: Environmental perception and representation are among the most critical tasks in automated driving. To meet the high demands on reliability from clients and the needs of safety standards, such as ISO 26262, there is a need for automated quantitative evaluation of perceived information. However, the typical evaluation methods currently being used, such as a comparison with Ground Truth (GT), is not feasible in the real world. Creating a substitute for GT with annotated data is not efficiently scalable due to the manual effort involved in evaluating the sheer number of scenarios, environmental conditions, etc. Hence, there is a need to automate the generation of data used as GT. This thesis focuses on a methodology to generate a substitute for GT data, named Pseudo Ground Truth (PGT), with a LiDAR sensor and to identify the precautions needed if this PGT is to be used in the development of perception systems. This thesis aims to assess the proposed methodology in a common scenario. The limitations with the LiDAR sensor are analyzed by performing a Systematic Literature Review (SLR) on available academic texts, conducting semi-structured interviews with experts from one of the largest heavy vehicle manufacturers in Europe, Scania CV AB and lastly implementation of an experimental algorithm to create a PGT. The main contributions are 1) a list of found limitations with the current LiDAR sensors from a SLR and semi-structured interviews and 2) a proposed methodology, which assesses the use of a LiDAR sensor as GT in a scenarios coupled with a set of precautions that has to be taken if the method were used in the development of new perception systems.