Visual Place Recognition (VPR) involves determining the geographic location of a photo based solely on its visual content. Recent advancements in Deep Learning (DL) have enabled the representation of images in high-dimensional spaces, where photos taken in the same location tend to cluster together, while images from different places are spread apart. This spatial organization makes it easier to predict locations by performing similarity searches against a database of known places. However, a key gap in current research is understanding the specific information retained in these image embeddings that allows for effective and reliable location prediction. Additionally, existing State-of-the-Art (SOTA) deterministic methods in VPR are unable to quantify the uncertainty of their predictions.