Heterogeneous Multi-Sensor Camera

Abstract: Today's security cameras typically capture either color images based on visible light or grayscale images based on both visible and near-infrared (NIR) light. This means that the performance is limited in situations where it is unclear which kind of image to capture. At dusk, for example, the visible light may not be sufficient to capture good-quality color images. On the other hand, capturing grayscale images at dusk means that useful color information is disregarded. Thus, there is a need for a hybrid-mode camera which can utilize the best of both the color and the grayscale images. In order to build a prototype camera which can achieve just this, a dichroic beam-splitter was placed in between the lens and the image sensor of a conventional camera. This way, the incident light is split up into its visible and NIR components which then can be detected separately by two image sensors. Different approaches on how to merge color and grayscale images into one superior image were investigated, where the basic idea was to extract the overall intensity from the color image and partly replace it with the intensity of the grayscale image. The prototype camera developed proved to outperform conventional security cameras in certain situations, such as in low light conditions as well as when the illumination is highly varying. While more development is needed, the technique looks promising overall and should offer new imaging capabilities to the security camera market.