Towards hardware accelerated rectification of high speed stereo image streams

Abstract: The process of combining two views of a scene in order to obtain depth information is called stereo vision. When the same is done using a computer it is then called computer stereo vision. Stereo vision is used in robotic application where depth of an object plays a role. Two cameras mounted on a rig is called a stereo camera system. Such a system is able to capture two views and enable robotic application to use the depth information to complete tasks. Anomalies are bound to occur in such a stereo rig, when both the cameras are not parallel to each other. Mounting of the cameras on a rig accurately has physical alignment limitations. Images taken from such a rig has inaccurate depth information and has to be rectified. Therefore rectification is a pre-requisite to computer stereo vision. One such a stereo rig used in this thesis is the GIMME2 stereo camera system. The system has two 10 mega-pixel cameras with on-board FPGA, RAM, processor running Linux operating system, multiple Ethernet ports and an SD card feature amongst others. Stereo rectification on memory constrained hardware is a challenging task as the process itself requires both the images to be stored in the memory. The FPGA on the GIMME2 systems must be used in order to achieve the best possible speed. Programming a system that does not have a display and for used for a specific purpose is called embedded programming. The purpose of this system is distance estimation and working with such a system falls in the Embedded Systems program. This thesis presents a method that makes rectification a step ahead for this particular system. The functionality of the algorithm is shown in MATLAB and using VHDL and is compared to available tools and systems.