Infrared and Visible Image Fusion with an Unsupervised Network

Abstract: Image fusion is a technique that aims to combine semantic information from different source images into a new synthesized image that contains information from both source images. It is a technique that can be useful in many different areas, such as reconnaissance, surveillance and medical diagnostics. A crucial aspect of image fusion is finding important features from the source images and preserving these in the fused image. A possible method to find and preserve the features could be to utilize deep learning. This thesis trains and evaluates an unsupervised network on two new datasets created for the fusion of visual near infrared (VNIR) and long wave infrared (LWIR) images. Feature representations obtained from a pre-trained network are implemented in the loss function, followed by training and evaluation of that model as well. Both deep learning models are compared with results obtained from a traditional image fusion method. The trained models performed well whereas the traditional method performed better when evaluating dataset 1. The deep learning models did perform better on dataset 2 which contained images captured in daylight and dusk conditions. The resulting fused images from the deep learning approaches demonstrated better contrast compared to the fused images obtained by averaging. The additional feature representations obtained from the pre-trained network did not improve the results on any of the datasets. An explanation for these results could be that the loss function already helps to preserve the semantic information in the features.