Real-world low-light image enhancement using Variational Autoencoders

Abstract: Low-light image enhancement is a hard task mainly due to the amount of noise and little information stored in the dark image. In this thesis project we develop a method for low-light image enhancement based on a Conditional Variational Autoencoder (CVAE). The CVAE is a deep learning model trained with a specific objective function (ELBO). A CVAE can be implemented as a variant of a U-Net utilizing skip connections. We train models on a dataset called `See in the dark' (SID), which contains aligned dark-bright image pairs. This thesis is mainly focusing on building models which accept an unprocessed RAW image input and predict a noise-free bright color image. Additionally we also consider alternative input data (sRGB instead of RAW) and the task of predicting a grayscale image (instead of color). Different neural-network architectures are implemented and compared, so are different strategies for merging image-patches. Different variants of the CVAE are compared, in particular a model only using a reconstruction loss is considered. Regarding the different tasks the results turn out as expected. Models with RAW input data perform better than the model with sRGB input. Models which predict color perform worse than the one predicting only luminance. The output images for the model predicting only grayscale have a bit more detail and have less artifacts. The specific type of interpolation (between predicted patches) is of significant importance in case the inputs are extremely noisy, otherwise the faster methods are good enough. Comparison of different types of CVAE models with the model trained only using reconstruction loss shows that the very type of loss-function is not of great importance for the results, but the CVAE models are holding up well. The overall results are rather a function of the network architecture and the type of input data. Overall we conclude that the CVAE model can successfully be applied to the task of real-world low-light image enhancement, for visual improvement, in case the input is not too noisy. In case of an extremely noisy input the results are not as good.