Deep Reinforcement Learning in Real-time Bidding

Abstract: Real-time bidding is getting increasingly popular for buying and selling online display advertisement. This has spurred a research interest into how to design optimal bidding algorithms, with advances during the last two to three years focusing heavily on reinforcement learning. This thesis focuses on creating bidding agent using recent innovations in combining reinforcement learning and deep learning, drawing heavily from a recent paper by Wu et al. (2018). However, the final algorithm presented in this thesis, called (Batch) Deep Reinforcement Learning to Bid (Batch-DRLB) deviates quite a bit from their algorithm. Batch-DRLB shows superior results to two simple benchmark algorithms and compares very well to current state-of-the-art algorithms. This project has been done in collaboration with Adform, which is one of the world's largest advertising technology companies, based in Copenhagen. They have provided fantastic support throughout the project. In addition to providing great resources for developing and testing the algorithm, they've provided continuous help in getting a better understanding of RTB and computational advertisement. The final algorithm is something like a thousand lines of code. Hence, I've chosen not to include it in here and have instead provided all of the code in a GitHub repository: https://github.com/Ostigland/dqn-rtb