Impact of Semantic Segmentation on OOD Detection Performance for VAEs and Normalizing Flow Models

Abstract:  To achieve a higher grade of reliability among deep learning models, OOD (Out-Of-Distribution) detection has become an increasingly more prominent research field. What OOD detection does is to make the model figure out if inputted data is data it is meant to be trained on, or if it is data from a new distribution, giving the model a sense of its own ignorance. Models regularly used in the OOD detection field are likelihood-based generative models. This is because, unlike discriminative models, these model the full data distribution, learning more about the data itself. To get more information and structure about the data, one can use semantic segmentation. Semantic segmentation is when all pixels in an image of a certain class are marked with corresponding values and the other unrecognized pixels are marked as zero. This thesis test the impact of using semantic segmentation as additional inputs to VAEs and normalizing flow models. The results show that semantic segmentation does have an impact on the likelihood given by the models and is, therefore, something worth investigating further.