Explainable Reinforcement Learning for Gameplay

Abstract: State-of-the-art Machine Learning (ML) algorithms show impressive results for a myriad of applications. However, they operate as a sort of a black box: the decisions taken are not human-understandable. There is a need for transparency and interpretability of ML predictions to be wider accepted in society, especially in specific fields such as medicine or finance. Most of the efforts so far have focused on explaining supervised learning. This project aims to use some of these successful explainability algorithms and apply them to Reinforcement Learning (RL). To do so, we explain the actions of a RL agent playing Atari’s Breakout game, using two different explainability algorithms: Shapley Additive Explanations (SHAP) and Local Interpretable Model-agnostic Explanations (LIME). We successfully implement both algorithms, which yield credible and insightful explanations of the mechanics of the agent. However, we think the final presentation of the results is sub-optimal for the final user, as it is not intuitive at first sight.