Integrating Neuro-Symbolic Reasoning into 3D Point Cloud Semantic Segmentation

Semantic segmentation of 3D point clouds is a fundamental problem in computer vision, particularly relevant to the digital preservation and automated analysis of cultural heritage. Recent deep learning models, such as Point Transformer~v3, have achieved state-of-the-art performance by leveraging attention mechanisms to capture both local geometry and long-range spatial relationships. However, purely data-driven approaches still struggle to generalize across geometrically complex or underrepresented classes and offer limited interpretability. This thesis explores a neuro-symbolic framework that integrates logical reasoning into the learning process of a transformer-based point cloud segmentation model. The proposed method combines the representational capacity of deep neural networks with the interpretability and domain awareness of Logic Tensor Networks (LTN), implemented through the LTNTorch library. Symbolic knowledge about architectural structures—such as coplanarity, verticality, and spatial relationships between classes—is encoded as first-order logic rules and translated into differentiable constraints that guide learning under a best-satisfiability objective. Paired with standard empirical risk minimization, this formulation provides a regularizing effect, encouraging predictions that are both data-consistent and logically coherent. Experiments are conducted on the ARCH dataset, a large-scale benchmark of annotated architectural point clouds. As a baseline, Point Transformer~v3 is trained under standard empirical risk minimization and achieves a mean Intersection-over-Union (mIoU) of 77.5 % on the SMV test scene and 71.8 % on the SMG scene, outperforming previous DGCNN-based benchmarks by over 20 percentage points. A detailed per-class and qualitative analysis revealed semantic inconsistencies in the predictions—for example, violations of spatial or structural coherence between neighboring elements—that persist even in high-performing categories. These inconsistencies motivated the introduction of symbolic priors to enforce domain-consistent reasoning during training. The neuro-symbolic variant integrates domain rules as trainable predicates, designed to assess whether structured symbolic knowledge can complement data-driven representations and promote logically consistent predictions. Overall, this work establishes a reproducible framework for evaluating neuro-symbolic reasoning in 3D point cloud segmentation, building upon a thorough assessment of a state-of-the-art model on the ARCH dataset, and outlining promising directions for explainable scene understanding in digital heritage.