Investigating the Integration of Detailed Features into 3D Garment Reconstruction through a Two-Stage Pipeline with Label Masks for Enhanced Parametric Modelling

Technological advancements in computer vision and machine learning are transforming garment design and reconstruction processes across various industries and domains, including fashion, virtual try-ons, and historical preservation. With developments in parametric modelling, garment reconstruction has evolved from basic shapes to highly detailed representations. Despite these advancements, current techniques often overlook essential detailed features of the garment, such as buttons, collars, hems and pockets, leading to a simplistic and generalized digital representation of the garment. This overlook causes a gap in achieving fully realistic digital garment models. To address this issue, this thesis, made in collaboration with the Korea Advance Institute of Technology (KAIST), investigates garment reconstruction through a novel approach that integrates detailed parametric modelling for top garments. Specifically, the project builds upon SPNet and the Garment-Pattern-Generator frameworks, adapting them to explore the potential of rendering garment details from single images. SPNet, known for its two-stage pipeline that predicts sewing patterns from single images by generating T-pose representations of the garments and inferring from them pattern parameters, is adapted to capture elaborate garment details. This requires significant modifications to the architecture, including the integration of label masks for each detail (i.e. buttons, collars, hems and pockets) in the pipeline and additional parameter predictions, aiming to advance garment reconstruction capabilities. The Garment-Pattern-Generator framework is a system originally designed to generate large-scale datasets of 3D garments through parametric templates and it operates by defining garment construction parameters in a JSON-based format. This framework allows the generation of diverse garment patterns, which can be simulated and rendered into 3D models. In this work, the framework is extended with a new parametric template and sets of parameters to include detailed garment features, thereby supporting more complex garment designs. Furthermore, the thesis uses the enhanced Garment-Pattern-Generator framework to create an experimental synthetic dataset of top garments with details such as buttons, collars, hems and pockets. This dataset serves to train and evaluate the modified SPNet pipeline, which employs high-resolution modelling tools such as Autodesk Maya and the Qualoth simulator to render the predicted detailed 3D garments and their sewing patterns from single images. The accuracy of garment reconstruction with detailed features is evaluated by comparing outcomes with the ground truth sewing patterns and 3D models of the input images. While the refined parametric framework demonstrates promise, the results of the predicted parameters indicate a need for further advancements to fully capture complex garment details. This thesis thus provides insights into the potentials and limitations of the proposed parametric modelling techniques for detailed garment reconstruction, setting the stage for further development in this evolving area of research.