A Comparative Study on the Importance of Image Resolution in Gesture Recognition

Abstract: Sign language translation applications could provide a whole new avenue of communication. However, translating sign language comes with challenges such as deriving and handling information from images which can be a difficult task for computers. To make such a service versatile it should be able to run on a mobile phone which means limited processing power and space capacity. This thesis aims to research if lowering the image quality is a viable way to decrease the processing power and space capacity needed, while keeping as much accuracy in the object detection step as possible. A skeleton tracking model was used for hand detection, where both accuracy and processing time was measured over several resolutions. The accuracy was measured by the mean average precision detailed in the COCO Keypoint Detection challenge [1] and the overall recall. The study found that the overall recall and mean average precision decreased with lower resolutions. However, for the highest resolutions the decrease in accuracy was small compared to the decrease for lower resolutions. The processing time also had a general downward trend when lowering the resolution. This study concludes that the method of lowering the resolution can be used to gain time and memory without a significant drop in accuracy.