Exploring the parameter space of Q-learning for faster convergence using Snake

Abstract: In this paper we explore the field of reinforcement learning which has proven to be successful at solving problems of random nature. Such problems can be video games, for example the classical game of Snake. The main focus of the paper is to analyze the speed, measured in Q-table updates, at which an agent can learn to play Snake by using Q-learning, specifically with a Q-table approach. This is done by changing a set of hyperparameters, one at the time, and recording the effects on training. From this, we were able to train the agent with only 225 000 Q-table updates, which took 7 seconds on a regular laptop processor, and achieve a high score of 52 (34 % cover of the grid). We were able to train the agent with only 100 000 updates but it came with reliability issues such as in some training sessions it did not perform well at all.