Characterizing Pose Uncertainty in Semantic Perception Pipelines : Leveraging semantic information to improve path planning in dynamic environments

Abstract: Mobile robots possess complex perception pipelines composed of visual and depth sensors which allow them to understand their location and the world around them and build models of this world. Real-time motion planning of mobile robots in complex environments requires the knowledge of the properties of the perception pipeline so that accurate, robust and safe robot operations can be performed. Uncertainties are often modelled at the mapping and pose estimation stage and the best guess gets presented in the final output. In motion planning, often times general uncertainty is assumed about the map/pose presented without actually considering information obtained during localization and mapping. This paper exploits the gap between estimation and planning, especially in the presence of uncertainty. By better representing uncertainty in estimation, we can make more informed decisions during planning. We build upon methods that estimate the localizability from a given pose in a previously mapped environment. This is achieved by, for each position in a grid map, accumulating a measure of the visual quality of all features that would be visible from there based on the information collected during the mapping phase. This grid map can then be queried by a motion planner, allowing the planner to trade off reaching the goal faster and making sure the robot can stay localized when selecting positions and orientations in the map. We develop a framework that also takes into consideration the semantic classes of the features. By labelling the features in the map with semantics, differentiation between features is possible, making sure areas with more reliable features are preferred in the planning phase