Targeted Improvement of a Deep Learning Object Detector Using Synthetic Training Data

Abstract: When working with object detection, the quality and quantity of the training data is often a recurrent bottleneck. This thesis proposes a technique of incrementally improving an object detector using synthetically rendered data. The current training data within the field of focus is limited, and creating new data comes with significant integrity and security risks. By creating synthetic images where rendered people are placed in different adapted environments, the possibility of performing a targeted improvement of an object detector was examined. First, the detection errors of an object detection model, trained on real data, were explored to identify weaknesses to target. Next, one of the found weaknesses was used to determine the content of the synthetic data. Furthermore, the synthetic data was generated, resulting in multiple datasets. Domain randomization techniques were used to optimize the data quality, and 3D graphics tools were used to generate bounding boxes. New models were then trained on a mix of real and synthetic data. Finally, the best synthetic model was extracted, and another detailed evaluation was performed. The results showed that the new model, trained on additional synthetic data, did not suffer any unwanted side effects in its predictions. In addition, the ratio between the real and synthetic training data was examined, showing that using twice the amount of real data compared to synthetic provided the best results. The performance of the synthetic model improved compared to the initial model, trained solely on real data, for three different test datasets. The detailed evaluation showed that the number of detection errors for the targeted weakness decreased significantly. In conclusion, the results showed great promise in using synthetically rendered image data combined with real data to improve an object detection model.