3D Privacy Masking using Monocular Depth Estimation

Abstract: This thesis strives to dive deeper within the area of Monocular Depth Estimation, approximating distance information from one single image using deep neural networks. It introduces a thorough evaluation and analysis of state-of-the-art depth estimation models regarding proposed aspects of relevance for downstream video applications, specifically in a surveillance domain. This leads to three custom data sets where two include ground truth depth data. Results on accuracy, temporal inconsistency, and range resolution is presented and analysed utilising the collected data sets, with selected metrics. It is concluded that the accuracy performance of the models, even though impressive, is also highly scene dependant. Regarding temporal inconsistency, which causes apparent video instability, it is concluded to be a prominent concern for typical downstream video applications that calls for further attention. This leads to a proposed minor post-processing step, with promising results. Furthermore, this thesis also presents a novel end-to-end algorithm referred to as ”3D Privacy Masking”. Privacy masking is a typical task in camera surveillance, where a certain region of the image scene needs to be anonymised. This functionality is here extended by including depth, such as that from monocular depth estimation, resulting in a depth aware privacy mask. Thereby, events in front of the mask as seen by the camera can still be observable. The suggested algorithm and proof-ofconcept application also includes a stabilising technique to account for sub-perfect depth data. Conclusively, this thesis showcases the potential of monocular depth estimation in downstream computer vision tasks, like that of 3D privacy masking, and proposes continued directions forward.