FPGA implementation of an undistortion model with high parameter flexibility and DRAM-free operation

Abstract: Computer Vision (CV) has become omnipresent in our everyday life and it’s starting to see more and more use in the industry. This movement creates a demand for ever more performant systems to keep up with the increasing demands in manufacturing speed and autonomous behaviours. Such computer vision (CV) systems need to run complex algorithms at real-time speed and sometime even in energy constrained systems. Thus efficient implementation of these algorithms are a must. One of those algorithm is the lens rectification algorithm (also sometime called undistortion algorithm) that is often one of the first algorithm to be used to correct for multiple imperfection that can occur in a camera and lens system. This algorithm has been implemented on FieldProgramable Gate Arrays (FPGAs) in past work but they either relied heavily on Dynamic Random Access Memory (DRAM) or used a subset of the full lens distortion model used by OpenCV and restricted themselves to small distortion amounts by having access to only parts of the image at a time. This thesis aims to create an open-source, DRAM-free FPGA implementation of the OpenCV lens rectification model of this algorithm, using the full 12-parameter model and allowing the use of any parameter which, to our knowledge, has not yet been implemented. To do so, a hybrid programming approach was taken meaning that both Hardware Descriptive Languages and High-Level Synthesis were used to arrive at the final implementation. The final implementation achieves 1300 frames per second with a sub-millisecond latency at a resolution of 320x240 on grayscale images.