Relocalization of Autonomous Agents Using Monocular Depth Estimation on PTZ Cameras

This thesis presents a systematic approach for calibrating a Pan-Tilt-Zoom network camera and integrating it with state-of-the-art object detection and depth estimation algorithms for distance measurement of autonomous agents. A calibration methodology was adopted to compute the intrinsic parameters of the camera across various zoom levels, addressing the non-linear variations induced by dynamic zoom adjustments. This calibration process is fundamental for establishing an accurate mapping between the three-dimensional world and the two-dimensional image plane of the camera. Subsequently, the study focuses on estimating the distance of autonomous agents by leveraging a dual-framework approach. Object detection is performed using YOLOv8, chosen for its balance between computational efficiency and detection accuracy in real-time applications. For depth computation, a monocular image-based technique is employed using the Apple Depth Pro model, which incorporates a Vision Transformer architecture to capture high-level contextual features and infer depth from a single frame. The integrated system exploits the complementary strengths of YOLOv8 and ViT-based depth estimation. The object detection component provides precise localization, while the depth estimation framework supplies spatial measurements, resulting in an accurate distance assessment to the target autonomous agent. The results indicate that the proposed methodology achieves robust performance in distance estimation, with notable improvements in detection accuracy and depth precision compared to traditional methods. In conclusion, the primary goal of this work and its main results is the relocalization of autonomous agents thanks to an external PTZ camera. This approach originated from the challenges posed by the Leonardo Drone Contest. Moreover, it holds significant promise for real-world applications, including warehouse logistics, precision agriculture, and smart city infrastructure.