Labelling Customer Actions in an Autonomous Store Using Human Action Recognition

Abstract: Automation is fundamentally changing many industries and retail is no exception. Moonshopis a South African venture trying to solve the problem of autonomous grocery storesusing cameras and computer vision. This project is the continuation of a hackathon heldto explore different methods for Human Action Recognition in Moonshop’s stores.Throughout the project a pipeline for data processing has been developed and two typesof Graph-Convolutional Networks, CTR-GCN and ST-GCN, have been implementedand evaluated on the data produced by this pipeline. The resulting scores aren’t goodenough to call it a success. However, this is not necessarily a fault of the models. Rather,there wasn’t enough data to train on and the existing data was of varying to low quality.This makes it complicated to justly judge the models’ performances. In the future, moreresources should be spent on generating more and better data in order to really evaluatethe feasibility of using Human Action Recognition and Graph-Convolutional Networksat Moonshop.