Off-road Driving with Deteriorated Road Conditions for Autonomous Driving Systems

Abstract: Recent studies on robustness of machine learning systems shows that today’s autonomous vehicles struggle with very basic visual disturbances such as rain or snow. There is also a lack of training data that includes off road scenes or scenes with different forms of deformation to the road surface. The purpose of this thesis is to address the lack of off-road scenes in current dataset for training of autonomous vehicles and the issue of visual disturbances by building a simulated 3D environment for generating training scenarios and training data for specific environments. The synthesised scenes is implemented using modern OpenGL, and we propose methods to synthesis rutting and the formation of potholes on road surfaces as well as rain and fog with a parameterized approach. \\ The generated datasets are tested through semantic segmentation using state of the art pretrained neural networks. The results show that the neural networks accurately identifies the road surface in in clear weather as long as the road surface is mostly coherent. The synthesised rain and fog decrease performance of the neural networks significantly. \\ Generating training data with the method presented in this thesis and incorporating it as part of the training data used in training neural networks for autonomous driving systems could be used to improve performance in certain scenarios. Specifically, it could improve performance in driving scenes with heavy road deformations, and in scenes with low visibility. Further research is needed to conclude that the data is useful, but the results generated in this thesis is promising.