On Optimization of Sequential Decision-Making in Customer Relationship Management using Deep Reinforcement Learning

Abstract: Customer relationship management (CRM) is a fickle but pivotal elementto the success of any business. Used correctly, it can not only yield higherrevenue and lower operational costs, but significantly boost customersatisfaction. Nonetheless, it can also be mismanaged—sacrificing thewell-being of customers for profitability. Industries have thereby beenflooded with a range of different heuristic strategies that aim to optimizeCRM. This thesis aims to instead study and optimize CRM using a datadrivenapproach, and present a framework that can readily incorporatecustomer well-being into the optimization process. More specifically: cana strategy that outperforms a business’ current strategy without any realworldimplications be derived using modern advances in reinforcementlearning? In this context, the lifetime value (LTV), i.e. net profit, of acustomer will be used as the objective function to optimize for.Using deep feed-forward neural networks, an artificial environmentmimicking typical customer behavior was attained. The model’s predictivecapabilities deviated merely a couple of percent from the true averagecustomer behavior seen in the data. This was further leveraged byan algorithm to obtain a business strategy through reinforcement learning.This novel algorithm is based on deep Q-networks, with furtherdomain-specific additions such as combined experience replay and doublelearning. The algorithmically derived business strategy theoreticallyoutperformed the current state-of-the-art business strategy by approximately100 percent in average 2-year LTV, and further outperformed aplethora of different business strategies.