Towards Adaptive Image Resolution for Visual SLAM on Resource-constrained Devices

Abstract: Today, a large number of devices with small form factors and limited resources are being integrated with processes to perform complex tasks such as localization and mapping. One example of this are headsets used for Extended Reality. These devices are expected to perform under changing conditions in the environment and in the available resources, which require sophisticated control policies. In this thesis project, we start investigating the feasibility of online control of the image resolution of the camera sensor used for Visual Localization, for the purpose of minimizing the requirements of the process without decreasing the performance. Specifically, we perform extensive experiments on two Visual Simultaneous Localization and Mapping systems and a Visual Odometry system on two platforms with limited resources to see how the performance metrics are affected by the image resolution. Moreover, we model the localization error of ORB-SLAM3 with the use of feature matching statistics and the camera velocity. Our experimental results show that savings in terms of the execution time of Visual Localization by adapting the image resolution is possible in some situations. But we did not find significant potential savings in terms of the power consumption of the devices. We also found that the feature matching statistics improve predictions about the localization error of ORB-SLAM3 in several situations compared to only using the camera velocity. But the results are limited to a set of known scenarios, which highlights the difficulty of the modelling problem. Nevertheless, this thesis provides valuable insights into how sensor parameters affect the performance of Visual Localization, and how the localization error relates to tracking statistics inside the localization process.