The ghost in the machine : Exploring the impact of noise in datasets used for graph-based action recognition

Abstract: Human action recognition is the task of classifying human movement and actions from video data. To benchmark different algorithms within the action recognition field, a common benchmark dataset, called NTU-RGB+D is used. However, this dataset is not without its issues as some samples contain data that is mistakenly captured as a human. In the context of this thesis, these are defined as ghost bodies. This thesis explores to what extent the accuracy of a state-of-the-art directed graph neural net, DGNN, is affected if trained without ghost bodies. The results suggest that the accuracy increases by 1.79 percentage points when ghost bodies are excluded during testing with an unofficial implementation of the DGNN. However, the results of the original DGNN could not be fully replicated which undermines the strength of the results. Despite this, given the importance of the NTU dataset within action recognition, we suggest considering a new benchmark dataset that takes ghost bodies into account. While the results of the study are not generalizable, the measured difference in recognition accuracy still points to the the necessity of looking deeper into the phenomenon of ghost bodies within action recognition.