Real-time Traffic Sign Detection and Classification : Evaluation of Image Processing performed on an FPGA-based platform

Abstract: As a part of the development of autonomous vehicles and advanced driver-assistance systems (ADAS), vision systems are used as a method for collection of extensive data about the surrounding world [1]. This data can thereafter be processed and information can be extracted. Due to the safety-critical nature of automotive applications, the image processing of the camera stream must be performed in real-time [2]. This thesis investigates how a system with real-time performance potential - an FPGA-based system - can be utilised to perform image processing applications. Specifically the thesis looks into the research area of traffic sign detection and classification. A literature study is presented, along with a proposed implementation of a Traffic Sign Detection and Classification (TSDC) system. The conclusion of the literature study is that many different methods have been tested previously but their performances are hard to compare. One of the most common approaches for FPGA-based implementations was chosen, due to its simplicity yet proven high accuracy by previous FPGA-based implementations. The approach - a colour thresholding and template matching - was partly implemented using the manufacturer Xilinx’s developing tool Vivado and High Level Synthesis (HLS). The entire system was never implemented due to lack of time. However, the colour thresholding part of the algorithm was implemented and provided good result with a throughput of 209 frames/s, very low hardware utilisation and a low power consumption of 0.016 J/frame. This was determined using Vivado’s Design Evalu-ation tools. A proof of concept was provided for the classification part of the system, that was never implemented on the platform, which showed that the classification part would likely constitute a performance bottleneck to the system. The detection and classification results proved that if there was a sign in the image it was found 96.0 % of all cases on previously unseen data, but of those where only 79.0 % classified as true positives. In addition to this 34.9 % of the previously unseen images not containing the searched-for sign, were a false positive. The conclusion of the thesis is that for a full system to be implemented, more of the tasks need to be performed on the FPGA, in order to have the potential to perform in real-time. One proposal to achieve this, is to implement a region of interest extraction, so only a single scale template match could be performed. However, given the classification results, it is probably a too simple classifier for the problem. Another conclu-sion is therefore that a more sophisticated classifier would be of interest to test instead.