Multi-View Vocabulary Trees for Mobile 3D Visual Search

Abstract: Mobile Visual Search (MVS) is a research field which focuses on the recognition of real-world objects by using mobile devices such as smart phones or robots. Current mobile visual search solutions achieve search results based on the appearance of the objects in images captured by mobile devices. It is suitable for planar structured objects such as CD cover images, magazines and art works. However, these solutions fail if different real objects appear similar in the captured images. To solve this problem, the novel solution captures not only the visual appearance of the query object, but uses also the underlying 3D geometry. Vocabulary Tree (VT) methods have been widely used to efficiently find the match for a query in the database with a large volume of data. In this thesis, we study the vocabulary tree in the scenario of multi-view imagery for mobile visual search. We use hierarchically structured multi-view features to construct a multi-view vocabulary trees which represent the 3D geometric information of the objects. Relevant aspects of vocabulary trees such as the shaping of trees, tf-idf weighting and scoring functions have been studied and incorporated in the multi-view scenario. The experimental results show that our multi-view vocabulary trees improve the matching and ranking performance of mobile visual search.