Safey-aware Adaptive Reinforcement Learning with Applications to Brushbot Navigation

Abstract: This thesis presents a safety-aware learning framework that employs an adaptivemodel learning method together with barrier certificates for systems withpossibly nonstationary agent dynamics. To extract the dynamic structure ofthe model, we use a sparse optimization technique, and the resulting modelwill be used in combination with control barrier certificates which constrainfeedback controllers only when safety is about to be violated. Under somemild assumptions, solutions to the constrained feedback-controller optimizationare guaranteed to be globally optimal, and the monotonic improvementof a feedback controller is thus ensured. In addition, we reformulate the(action-)value function approximation to make any kernel-based nonlinearfunction estimation method applicable. We then employ a state-of-the-artkernel adaptive filtering technique for the (action-)value function approximation.The resulting framework is verified experimentally on a brushbot,whose dynamics is unknown and highly complex.