Behavioral Targeting in E-Commerce Through the Use of Reinforcement Learning

Abstract: E-commerce websites often seek to maximize sales by employing sales agents to talk on the phone with selected visitors and assist them in making a purchase, and in order to make the most out of this investment, visitors must be targeted with contact offers intelligently. One approach to designing a targeting strategy is to have human experts define sets of rules that decide how to act, however, a more automated approach would be desirable. This thesis investigates a reinforcement learning approach to generating a targeting strategy. More specifically, the batch reinforcement learning algorithm fitted Q-iteration (FQI) is applied to clickstream data from a large E-commerce website to generate a targeting strategy, which is then evaluated both on historical data and in the live environment, and its performance is compared to the rule based system currently in use. Results show that the strategy generated by the FQI algorithm produces total conversion rates similar to that of the rule based system, however, it also shifts more of the sales towards the sales agents, which is undesirable.