Animation of humanoid characters using reinforcement learning

Abstract: Procedural animations are still in its infancy, and one of the techniques to create such is using Reinforcement Learning. In this project, swimming animations are created using UnityML version 0.6 with their Reinforcement Learning training agents, using the policy PPO, created by OpenAI. A humanoid character is placed in a simulated water environment and propels itself forward by rotating its joints. The force created depends on the joints mass and the scale of the rotation. The animation is then compared to a swimming animation created using movement capture data. It is concluded that the movement capture data animation is significantly more realistic than the one created in this project. The procedurally created animations displays many of the typical issues with reinforcement learning such as jittering and non-smooth motions. While the model is relatively simple, it is not possible to avoid these issues completely with more computational power in the form of a more complex model with more Degrees of Freedom. It is however possible to finetune the animations with the improvements listed at the end of the discussion.