Data augmentation using military simulators in deep learning object detection applications

Abstract: While deep learning solutions have made great progress in recent years, the requirement of large labeled datasets still limit their practical use in certain areas. This problem is especially acute for solutions in domains where even unlabeled data is a limited resource, such as the military domain. Synthetic data, or artificially generated data, has recently attracted attention as a potential solution for this problem. This thesis explores the possibility of using synthetic data in order to improve the performance of a neural network aimed at detecting and localizing firearms in images. To generate the synthetic data the military simulator VBS3 is used. By utilizing a Faster R-CNN architecture multiple models were trained on a range of different datasets consisting of varying amounts of real and synthetic data. Moreover, the synthetic datasets were generated following two different philosophies. One dataset strives for realism while the other foregoes realism in favor of greater variation. It was shown that the synthetic dataset striving for variation gave increased performance in the task of object detection when used in conjunction with real data. The dataset striving for realism gave mixed results.