Fault Detection and Diagnosis for Automotive Camera using Unsupervised Learning

Abstract: This thesis aims to investigate a fault detection and diagnosis system for automotive cameras using unsupervised learning. 1) Can a front-looking wide-angle camera image dataset be created using Hardware-in-Loop (HIL) simulations? 2) Can an Adversarial Autoencoder (AAE) based unsupervised camera fault detection and diagnosis method be crafted for SPA2 Vehicle Control Unit (VCU) using an image dataset created using Hardware-inLoop? 3) Does using AAE surpass the performance of using Variational Autoencoder (VAE) for the unsupervised automotive camera fault diagnosis model? In the field of camera fault studies, automotive cameras stand out for its complex operational context, particularly in Advanced Driver-Assistance Systems (ADAS) applications. The literature review finds a notable gap in comprehensive image datasets addressing the image artefact spectrum of ADAS-equipped automotive cameras under real-world driving conditions. In this study, normal and fault scenarios for automotive cameras are defined leveraging published and company studies and a fault diagnosis model using unsupervised learning is proposed and examined. The types of image faults defined and included are Lens Flare, Gaussian Noise and Dead Pixels. Along with normal driving images, a balanced fault-injected image dataset is collected using real-time sensor simulation under driving scenario with industrially-recognised HIL setup. An AAE-based unsupervised automotive camera fault diagnosis system using VGG16 as encoder-decoder structure is proposed and experiments on its performance are conducted on both the selfcollected dataset and fault-injected KITTI raw images. For non-processed KITTI dataset, morphological operations are examined and are employed as preprocessing. The performance of the system is discussed in comparison to supervised and unsupervised image partition methods in related works. The research found that the AAE method outperforms popular VAE method, using VGG16 as encoder-decoder structure significantly using 3-layer Convolutional Neural Network (CNN) and ResNet18 and morphological preprocessings significantly ameliorate system performance. The best performing VGG16- AAE model achieves 62.7% accuracy to diagnosis on own dataset, and 86.4% accuracy on double-erosion-processed fault-injected KITTI dataset. In conclusion, this study introduced a novel scheme for collecting automotive sensor data using Hardware-in-Loop, utilised preprocessing techniques that enhance image partitioning and examined the application of unsupervised models for diagnosing faults in automotive cameras.