Evaluation of machine learning methods to predict payment preferences

Abstract: The last couple of years machine learning has seen a renaissance, with Artificial Neural Networks in particular rising to prominence. The technology is being adopted by more and more businesses, with varying degrees of success. Klarna has already been experimenting with machine learning to predict payment preferences, however currently a hybrid between ad-hoc rules and a random forest model is being used in production. This report aims to find out if a pure machine learning algorithm can outperform a hybrid system for this purpose. To achieve this, four methods were tested; Random Forest, Artificial Neural Net- work, Support Vector Machine and Logistic Regression model. Three of these models outperformed the model in production. Best of these were the Artificial Neural Network which, with a cutoff threshold designed to achieve the same precision, achieved 10 percentage points higher recall. By combining the probabilities produced by an Artificial Neural Network and a Random Forest, even better results could be achieved. That method achieved 11.5 percentage points higher recall than production results with the same precision. It could be shown that the two methods had different strengths and were good at classifying different examples.