Deep learning for identification of figurative elements in trademark images using Vienna codes

Abstract: Labeling of trademark images with Vienna codes from the Vienna classification is a manual process carried out by domain experts, which enables searching trademark image databases using specific keywords that describe the semantic meaning of the figurative elements. In this research, we are investigating how application of supervised learning algorithms can improve and automate the manual process of labeling of new un-labeled trademark images. The successful implementation of deep learning algorithms in the task of computer vision for image classification has motivated us to investigate which of the supervised learning algorithms performs better trademark image classification. More specifically, to solve the problem of identification of figurative elements in new un-labeled images, we have used multi-class image classification approach based on deep learning and machine learning. To address this problem, we have generated a unique benchmarking dataset composed of 14,500 unique logos extracted from the European Union Intellectual Property Office Open Data Portal. The results after executing a set of controlled experiments on the given dataset indicate that deep learning models have overall better performance than machine learning models. In particular, CNN models reach better accuracy and precision, and significantly higher recall and F1 score for shorter training times, compared to recurrent neural networks such as LSTMs and GRUs. From the machine learning models, results indicate that Support Vector Machines have higher accuracy and overall better performance time compared to Decision Trees, Random Forests and Naïve Bayes models. This study shows that deep learning models can solve the problem of the labeling of trademark images with Vienna codes, and that can be applied by Intellectual Property Offices in real-world application for automation of the classification task which is carried out manually by the domain experts.