Understanding the needs of Image Retrieval for Visual Localization

Visual Localization is a key task in autonomous systems and robotics, consisting in the estimation of camera poses given their captures. Despite different approaches exist, they always rely on comparing the query image to reference images with known poses. To improve efficiency, relevant images to be matched are selected with retrieval pipelines. This work focuses on the impact of retrieval algorithms in Visual Localization. Firstly, current retrieval methods are benchmarked to determine the current state of the art on the task. Secondly, the needs of retrieval are investigated by selecting images using the known poses on different criteria. Lastly, different tasks related to Visual Localization are explored. The experiments highlight the limits of current approaches, as well as their margins of improvement and the future working directions to remove the retrieval performance bottleneck in Visual Localization.