Resource Optimal Neural Networks for Safety-critical Real-time Systems

Abstract: Deep neural networks consume an excessive amount of hardware resources, making them difficult to deploy to real-time systems. Previous work in the field of network compression lack the explicit hardware feedback necessary to control the resource constraints imposed by such systems. Furthermore, when the system under discussion is safety-critical, additional constraints must be enforced to make sure that acceptable safety levels are achieved. In this work, we take a reinforcement learning approach with which we evaluate three different compression actions: filter pruning, channel pruning and Tucker decomposition. We found that channel pruning was the most consistent one as it satisfied the constraints specification on five of six test scenarios while providing compression and acceleration rates of 10-30% across most resource metrics. By further optimizing the networks with TensorRT, we managed to improve the resource efficiency of the reference networks by up to 6×.