Deep Reinforcement Learning on Social Environment Aware Navigation based on Maps

Abstract: Reinforcement learning (RL) has seen a fast expansion in recent years of its successful application to a range of decision-making and complex control tasks. Moreover, deep learning offers RL the opportunity to enlarge its spectrum of complex fields. Social Robotics is a domain that involves challenges like Human-Robot Interaction which bears inspiration for development in deep RL. Autonomous systems demand a fast and efficient environment perception so as to guarantee safety. However, while being attentive to its surrounding, a robot needs to take decisions to navigate optimally and avoid potential obstacles. In this thesis, we investigate a deep RL method for mobile robot end-to-end navigation in a social environment. Using the observation collected in a simulation environment, a convolutional neural network is trained to predict an appropriate set of discrete angular and linear velocities for a robot based on its egocentric local occupancy grid map. We compare a random learning way to a curriculum learning approach to ameliorate speed convergence during training. We divide the main problem by analysing separately end-to-end navigation and obstacle avoidance in static and dynamic environments. For each problem, we propose an adaptation that aims to improve the surrounding awareness of the agent. The qualitative and quantitative evaluations of the investigated approach were performed in simulations. The results show that the end-to-end navigation map-based model is easy to set up and shows similar performance as a Model Predictive Control approach. However, we discern that obstacle avoidance is harder to translate to a deep RL framework. Despite this difficulty, using different RL methods and configurations will definitely help and bring ideas for improvement for future work.