Promoting Exploration in Reinforcement Learning through Surprise-Based Intrinsic Motivation

Abstract: Reinforcement learning is a subset of machine learning that has received widespreadattention in the past decade due to numerous notable achievements, especially frommethods known as deep reinforcement learning. However, many outstandingchallenges still remain - one of which being that of learning in environments withsparse reward signals. Various approaches inspired by the concept of intrinsicmotivation from behavioural psychology have been proposed as ways of overcomingthis difficulty. This report proposes a new formulation of surprise-based intrinsicmotivation in reinforcement learning that can be implemented in terms of thealternating direction method of multipliers - an established algorithm for solvingconvex optimisation problems. This formulation is evaluated on an examplesparse-reward task where it is found that, despite some promising results, it fails togive a significant improvement over simpler established methods