Narrow Pretraining of Deep Neural Networks : Exploring Autoencoder Pretraining for Anomaly Detection on Limited Datasets in Non-Natural Image Domains

University essay from Linköpings universitet/Medie- och Informationsteknik; Linköpings universitet/Tekniska fakulteten

Abstract: Anomaly detection is the process of detecting samples in a dataset that are atypical or abnormal. Anomaly detection can for example be of great use in an industrial setting, where faults in the manufactured products need to be detected at an early stage. In this setting, the available image data might be from different non-natural domains, such as the depth domain. However, the amount of data available is often limited in these domains. This thesis aims to investigate if a convolutional neural network (CNN) can be trained to perform anomaly detection well on limited datasets in non-natural image domains. The attempted approach is to train the CNN as an autoencoder, in which the CNN is the encoder network. The encoder is then extracted and used as a feature extractor for the anomaly detection task, which is performed using Semantic Pyramid Anomaly Detection (SPADE). The results are then evaluated and analyzed. Two autoencoder models were used in this approach. As the encoder network, one of the models uses a MobileNetV3-Small network that had been pretrained on ImageNet, while the other uses a more basic network, which is a few layers deep and initialized with random weights. Both these networks were trained as regular convolutional autoencoders, as well as variational autoencoders. The results were compared to a MobileNetV3-Small network that had been pretrained on ImageNet, but had not been trained as an autoencoder. The models were tested on six different datasets, all of which contained images from the depth and intensity domains. Three of these datasets additionally contained images from the scatter domain, and for these datasets, the combination of all three domains was tested as well. The main focus was however on the performance in the depth domain. The results show that there is generally an improvement when training the more complex autoencoder on the depth domain. Furthermore, the basic network generally obtains an equivalent result to the more complex network, suggesting that complexity is not necessarily an advantage for this approach. Looking at the different domains, there is no apparent pattern to which domain yields the best performance. This rather seems to depend on the dataset. Lastly, it was found that training the networks as variational autoencoders did generally not improve the performance in the depth domain compared to the regular autoencoders. In summary, an improved anomaly detection was obtained in the depth domain, but for optimal anomaly detection with regard to domain and network, one must look at the individual datasets.

  AT THIS PAGE YOU CAN DOWNLOAD THE WHOLE ESSAY. (follow the link to the next page)