Real Time Gym Activity Detection using Monocular RGB Camera

Abstract: Action detection is an attractive area for researchers in computer vision, healthcare, physiotherapy, psychology, and others. Intensive work has been done in this area due to its wide range of applications such as security surveillance, video tagging, Human-Computer Interaction (HCI), robotics, medical diagnosis, sports analysis, interactive gaming, and many others. After the deep learning booming results in computer vision tasks like image classification, many researchers have tried to extend the success of deep learning models to video classification and activity recognition. The research question of this thesis is to study the use of the 2D human poses extracted by a DNN-based model from RGB frames only, for the online activity detection task and comparing it with the state of the art solutions that utilize the human 3D skeletal data extracted by a depth sensor as an input. At the same time, this work showed the importance of input pre-processing and filtering on improving the performance of the online human activity detector. Detecting gym exercises and counting the repetitions in real-time using the human skeletal data versus the 2D poses have been studied in-depth in this work. The contributions of this work are as follows: 1) generating RGB-D dataset for a set of gym exercises, 2) proposing a novel real-time skeleton-based Double Representational RNN (DR-RNN) network architecture for the online action detection, 3) Demonstrating the ability of the proposed model to achieve satisfiable results using pose estimation models applied on RGB frames, 4) introducing a novel learnable exponential filter for the online low latency filtering applications.