Control Method for an Automated Forest Machine Based on Deep Reinforcement Learning

Abstract: An automated forest machine was designed in order to improve the working environment of today’s forest machine operators. In order to realize the autonomous control of the forest machine, model-based methods such as A' and dynamic window were used in previous projects. Considering the limited performance and modeling difficulty of the above methods, this thesis explores model-free control methods based on different Deep Reinforcement Learning algorithms, which are Deep Q-Network and Proximal Policy Optimization. The new controllers realize prediction and control based on the real-time position and the coordinate of the target point. The machine can reach the single target point after training. Besides the part of the controller, a spring-damper system is also implemented on the simulated model in order to increase the stability of the machine. The results from tests show that the controller based on Proximal Policy Optimization has quicker training speed, more stable validation results, and less offset following the global path compared with the controller based on Deep Q-Network. The machine can be controlled successfully from the initial position to the target point. The spring-damper system is also implemented and optimized on the joint of the pendulum arm of the simulated model. The results from speed bump tests indicate that the machine has a more stable movement with the new spring-damper system.