3D RECONSTRUCTION OF THE COLONIC MUCOSA FROM MONOCULAR ENDOSCOPIC VIDEO FOR UNOBSERVED AREA QUANTIFICATION

Colorectal cancer remains one of the leading causes of cancer mortality. Colonoscopy is the clinical gold standard for detecting and removing precancerous polyps because it provides direct visualization of the colonic mucosa. Nevertheless, the field of view is limited and the colon’s geometry is complex, so relevant areas can remain unobserved during routine inspection. This thesis introduces a pipeline that reconstructs a three-dimensional model of the inner colonic surface from monocular endoscopic video, with the aim of measuring where and how much mucosa was not seen. The method fuses RGB frames, predicted depth, and estimated camera pose through a TSDF-based reconstruction module to produce dense, anatomically faithful meshes. Depth is inferred using foundation depth models (Depth Anything, Video Depth Anything, DepthPro) and a fine-tuned network trained on the SimCol3D dataset; poses are estimated with a bimodal deep learning–based model trained on SimCol data. Reconstructions were generated from 15 virtual colonoscopies under different configurations, and an ablation study compared meshes built from estimated inputs against a GT-depth + GT-pose baseline. Three cases were evaluated: (i) estimated depth + GT pose, (ii) GT depth + estimated pose, and (iii) both estimated. Reconstructed meshes were aligned to the GT baseline, and performance was assessed using the symmetric Chamfer-L1 distance and mesh-to-mesh overlap at tolerance τ = 4 mm. (i) C-L1 = 4.50 mm ; mesh-to-mesh overlap = 82.3%. (ii) C-L1 = 38.13 mm ; mesh-to-mesh overlap = 35.7%. (iii) C-L1 = 41.18 mm ; mesh-to-mesh overlap = 27.9%. Missing regions are identified by closing reconstructed meshes with the Poisson Surface Reconstruction method and comparing the closed mesh to the original one. The detected missing regions are analyzed for their extent and size. Across all acquisitions, the percentage of unobserved surface averages 19.6% ± 1.9%, with unobserved areas clustering around deep folds and sharp flexures. To bridge simulation and reality, a hybrid dataset is acquired on a silicone colon phantom. An Olympus EVIS EXERA III endoscope captures fisheye video sequences, while an NDI AURORA electromagnetic tracker records six-degree-of-freedom trajectories of the endoscope tip within the operative field. RGB frames are paired with depth maps estimated using the Depth-Anything model, providing a multimodal dataset linking visual and spatial data. Some limitations remain: foundation depth models underperform on endoscopic video due to domain gaps (fluids, specular mucosa, low texture) and monocular scale ambiguity; colon deformability is treated as quasi-rigid; and the pipeline is not yet real-time. Despite these constraints, the framework produces smooth, anatomically coherent reconstructions, quantifies unseen mucosa with a simple metric, and shows clear gains from domain-specific fine-tuning. These results establish a practical basis for spatially aware analysis of colonoscopy, support objective coverage reporting, and lay the groundwork for future real-time clinical deployment.