Reassembling the Past: Techniques and Dataset to Improve 3D Reconstruction of Fragmented Artifacts for Cultural Heritage

Fragmented 3D objects’ computational reassembly is an innovative field of research with widespread applications, in particular within the cultural heritage sector for museums engaged in preservation, academic study, and public display of artifacts, but also for medical purposes. This thesis work, undertaken in collaboration with the Museo Egizio in Turin, investigates the potential for high-level computational techniques to aid in the reconstruction of archaeological artifacts, enhancing scholarly interpretation as well as accessibility. Restoring broken artifacts not only brings them to their former shape but also provides new opportunities for research, conservation, and exhibition. However, the work is compounded by heterogeneity and diversity of fragments as well as the absence of standardised data sources. This thesis project was initially undertaken in collaboration with another student, exploring valid solutions for fragment reassembly. The overall research project was then split into two independent paths to focus on different aspects of the problem. The present work focuses on evaluating techniques for enhancing the performance of an existing deep learning model in the context of fragment reassembly, while the other work focuses on a custom pipeline for a different approach. The devised work explores techniques for the computer-aided reconstruction of 3D objects from shattered fragments using existing, state-of-the-art deep learning models. A thorough review of the state of the art is presented, covering older techniques that use geometric, topological, and heuristic algorithms, as well as a detailed approach to neural network-based models. The methodological framework of the reported work consists of two phases. The first phase is the identification and preparation of a suitable dataset, an important step given that datasets specifically prepared for fragment reassembly are few. Adaptation and preprocessing methods are proposed to ensure that data are adequate for facilitating training and testing of machine learning algorithms. The second phase focuses on refining, tuning and training the best state-of-the-art model, PuzzleFusion++, for maximising accuracy and efficiency in object reassembly and fragment matching. Particular attention is given to architectural design, loss functions, and evaluation metrics with the aim of overcoming current benchmarks set by past work. Performance results of the model implemented, relative to PuzzleFusion++, are presented. Quantitative measures along with qualitative visualisations of reconstructed objects are discussed, showcasing the ability of the model to address varying degrees of fragmentation and complexity. The discussion section highlights the observed differences in results, taking into consideration model limitations, particularly for specific object categories. The thesis work contributes to the field of automated 3D object reassembly since it introduces a variation of a model that can attain competitive performance relative to the state of the art Perhaps more significantly, the work done highlights the capability of machine learning methods to aid cultural heritage institutions, such as the Museo Egizio, in the restoration and interpretation of fragmented artifacts. Directions for future research are also outlined, including data set growth, multimodal information fusion such as texture and material features, and explorations of additional cutting-edge neural structures for enhancing reassembly precision and appropriateness.