Social Distancing AI: Using super-resolution to train an object detection model on low resolution images

Abstract: This paper addresses the following hypothetical situation: suppose that we want to train an object detection algorithm but that only low resolution data is available. As a tentative solution to this problem, this paper suggest to first super-resolve the low resolution data to obtain higher resolution data that is then fed to the object detection algorithm. Super-resolution is a technique that takes a lower resolution image and “paints” in extra details in a very convincing way to produce a higher resolution output. To evaluate this approach, this paper trains the object detection model on to separate datasets: 4x down-sampled and subsequently super-resolved images and the original images. For super-resolution we use ESRGAN which is based on RaGAN. Quantitative comparison suggests that there is little difference between the two networks and thus provides some support for the approach explored in this paper. Qualitative comparison indicates that this technique only works in certain cases–when super-resolution results in images that can properly be annotated. If the contrast between the objects of interest and the background is great enough, then super-resolution will result in much clearer images that are easily annotated. In this particular scenario, using super-resolution to train object detection algorithms represents a feasible solution to the hypothetical situation assessed in this paper. This approach is tested in the context of detecting social distancing violations in restaurants using areal footage. This is of high relevance in 2020 due to the ongoing Coronavirus pandemic.