Uncertainty-Aware Semi-Supervised Learning for Neurosurgical Navigation

Accurate, real-time segmentation of different anatomical structures is essential for assisting surgeons during complex neurosurgical procedures. While pre-operative study often involves the use of MRI and other imaging techniques, neurosurgical navigation provides real-time assistance relying solely on video data. This is needed for identifying critical areas with high precision, as the relative positions of soft tissues can change during the procedure, and errors in anatomical identification can directly impact patient safety and procedural success. These scenes often involve subtle structures, such as arterial branches or aneurysms, that may only become visible partway through a procedure. Training models for this task requires dense, per-pixel annotations, which are labor-intensive and time-consuming, limiting the availability of fully labeled data in medical contexts. This work presents a novel framework for semi-supervised learning applied to segmentation of neurosurgical scenes that leverages both labeled and unlabeled data, to improve segmentation accuracy and reliability in a setting with a highly limited dataset. Central to this approach is an uncertainty-aware pseudo-labeling method, which uses Monte Carlo Dropout to compute an original metric called Semantic Spatial Uncertainty. This metric predicts the reliability of model outputs even without ground truth labels, ensuring that only high-quality pseudo-labels are selected, allowing the use of unlabeled data to improve performance. The model architecture combines a Vision Transformer backbone with a DeepLabV3+ segmentation head, where Monte Carlo Dropout enables estimation of prediction uncertainty, allowing Semantic Spatial Uncertainty to guide the selection of reliable pseudo-labels. Using adaptive, class-specific criteria, the framework prioritizes underrepresented classes and includes only the most reliable pseudo-labels. The approach also involves dynamic criteria adjustments and continuous tracking of uncertainty, enabling the model to iteratively refine pseudo-labels while filtering out lower-quality predictions. This continuous refinement enhances the model’s robustness and adaptability, particularly in challenging surgical environments. Experimental results show that this method increases segmentation accuracy for the most underrepresented classes, effectively addressing the constraints posed by limited annotations. Although demonstrated in neurosurgical navigation, this framework offers a scalable solution for medical image segmentation, with potential applications across other computer vision segmentation tasks. By producing consistent and reliable predictions, this work contributes an effective approach for advancing segmentation performance in data-constrained environments, reducing dependency on extensive manual annotations and enhancing accuracy in a wide range of practical applications.