Evaluation of Natural Language Processing and Machine Learning Tools for the Automation of the Customer Service Task

Abstract: Customer service is an integral part of every modern company. Nowadays, almost any company can offer their customers extended means of communication, which considerably change customers’ concept of speed and convenience, resulting in higher expectations of the quality of the delivered customer service. These growing expectations makes companies evaluate Artificial Intelligence technologies for enhancing customer service processes. The thesis aims to evaluate Natural Language Processing and Machine Learning tools for automating the process of dispute handling in the customer service area. Two main types of models are implemented to serve this purpose. The first one is based on the Bag-of-Words representation of text data, and the second involves the state-of-the-art Bidirectional Encoder Representations from Transformers model. The task is a multiclass classification with a set of text and combined text and non-text features. The experiment shows that the state-of-the-art Bidirectional Encoder Representations from Transformers model outperforms the models using the Bag-of-Words representation reaching 78.4% Accuracy on a set of text features. The best performance on a set of text representations combined with non-text features is 84.4% for the Neural Network model combining the fine-tuned DistilBERT with the Neural Network trained on solely non-text features. Possible use of the model, if implemented in production, could be a helping tool for customer service agents in their everyday activities supplying them with a suggestion on a closing reason for each dispute and, thus, enabling faster dispute closing or a sanity check tool, which tracks the correctness of a closing reason for each dispute and flags any contradictory cases