Detecting Defective Rail Joints on the Swiss Railways with Inception ResNet V2 : Simplifying Predictive Maintenance of Railway Infrastructure

Abstract: Manual investigation of railway infrastructure is a labor-intensive and time-consuming task, and automating it has become a high priority for railway operators to reduce unexpected infrastructure expenditure. In this thesis, we propose a new image classification approach for classifying defect and non-defective rail joints in image data, based on previous fault detection algorithms using object detection. The rail joints model is to our knowledge a world first, with the vast majority of research into applying computer vision for rail defect detection focusing mainly on the rail tracks and sleepers. Our new image classification models are based on the widely popular Inception ResNet V2, which we fine-tune and compare against a counterpart trained using self-supervision. Additional comparisons are performed against the Faster R-CNN object detector that has had successes with rail tracks and sleepers at the Swiss Federal Railways, as well as against the novel transformer-based DETR architecture. The research has used an in-house object detection annotated dataset from the Swiss railways, recorded in the context of predictive rail maintanance, with rail joints labeled as either defective, or non-defective. Our proposed image classification approach, using either a pre-trained and then fine-tuned, or self-supervised CNN, uses the bounding boxes in a dataset originally intended for object detection, to perform an expanded crop of the images around the rail joint before feeding it to the neural network. Our new image classification approach significantly outperforms object detection neural networks for the task of classifying defective and non-defective rail joints, albeit with the requirement that the rail joint has to be identified prior to classification. Furthermore, our results suggest that the trained models classify defective joints in the test set more consistently than human rail inspectors. The results show that our proposed method can achieve practical performance on unseen data, and can practically be applied to real-life defect detection with high precision and recall, notably on the railways operated by Swiss Federal Railways, SBB CFF FFS.