A Study on Fault-tolerance of Deep Neural Networks for Embedded Systems

Abstract: Deep learning is replacing many traditional data processing methods in computer vision, speech recognition, natural language processing and many more diverse end applications. Until only a few years ago, using deep learning networks for inference required large amount of computational resources such as memory, processing power and energy. It was not trivial to deploy the computationally-expensive deep neural networks on embedded devices with limited capabilities. In recent years however, deep learning is finding its way through the world of embedded devices. Embedded systems such as Internet of Things (IoT) devices, phones and even components in cars are being equipped with deep neural networks. This raises interesting challenges for both embedded designers and deep learning scientists to close the gap between these two domains. In this thesis work, some challenges involved in deploying deep learning for embedded systems were discussed, as well as some of the available solutions and frameworks. Moreover, focusing on the safety and fault-tolerance aspects of embedded systems, tolerance of deep neural networks against faults was investigated using an experiment-based research strategy. A fault injection framework was designed and implemented that targeted deep neural networks defined using PyTorch. The framework developed was used to perform fault injection experiments on a small deep learning network. It was found how faults have various impacts on the accuracy of the neural network depending on the type of layer targeted by the faults. Worst-case faults were identified and several architectural modifications on the deep neural network were examined to improve the fault tolerance of the neural network under study.