Inertial-aided EKF-based Structure from Motionfor Robust Real-time Augmented Reality

Abstract: The aim of this project was to develop a system that enables the overlay of computer graphics in a video sequence recorded by a moving camera. To do this, the camera’s position relative different landmarks in the picture needs to be estimated, a problem commonly referred to as “Structure from Motion” within the Computer Vision community, or “Monocular SLAM” within the robotics community. The system should be robust in the sense that it can handle partial occlusions and dynamic environments, as well as large open spaces without much texture in the image. To enable this, an IMU-unit is used to complement the visual input from the camera. The system is based on previous work by Civera et al. and uses the Extended Kalman Filter (EKF) to fuse the sensor inputs. A 1-point RANSAC method is used to efficiently detect and discard outliers in the sensor data. Inverse depth parameterization is used to enable the use of landmarks far away from the camera. The IMU used in the project is developed by X-IO Technologies, and uses an on-board algorithm developed by Madgwick to determine a precise and drift-free orientation. The system is evaluated using multiple video sequences recorded in a setting similar to where the system is intended to be used. Results indicate that the IMU really helps the system to differentiate ambiguities between translational and rotational movements, as well as keeping the system stable during smaller occlusions. Some cases where the system often fails are also identified. The performance of the system is evaluated, as well as how some differ