Enhancing End-to-End Multiple Object Tracking with Efficient Propagation Pre-Training and Complementary Long-range Tracklet Re-Identification

Tracking-by-propagation has emerged as a promising paradigm in Multi-Object Tracking (MOT), leveraging feature propagation across frames to maintain object identities within an end-to-end architecture. Existing approaches, such as SambMOTR, have demonstrated impressive performance, often rivaling or surpassing tracking-by-detection methods, all while utilizing a single fully trainable architecture based on DETR-like detectors. However, their effectiveness is limited by several important challenges. Jointly training a strong pre-trained detector and a propagation module from scratch often disrupts the detector's initialization, resulting in performance degradation. Additionally, tracking-by-propagation struggles to maintain tracklet consistency during long occlusions, leading to identity switches and deteriorating tracking performance. To address these limitations, we propose SambaMOTRv2, a novel framework that enhances tracking-by-propagation performance by integrating within SambaMOTR a memory- and time-efficient pre-training strategy for the propagating parts of the algorithm and a Samba-based re-identification system. Our pre-training approach alleviates the initialization issue by ensuring strong training of the propagation module, enabling end-to-end tracking-by-propagation to surpass the performance of the baseline method after joint training. The introduction of a long-range tracklet re-identification module allows the model to merge fragmented tracklets across prolonged occlusions, recovering tracklets from occlusions up to 300% longer than those in the previous method. Unlike prior approaches, our re-identification module performs online tracklet re-identification and merging, trained using video-level positive and negative pairs, rather than keyframe reference-frame pairs. Experimental results show that the integration of the implemented methods leads to a significant performance boost, allowing our model to outperform previous state-of-the-art methods on complex datasets such as DanceTrack and SportsMOT. These results highlight the effectiveness of our approach in addressing key challenges in tracking-by-propagation and substantially improving both tracking accuracy and identity preservation.