Garment3DGen: 3D Garment Stylization and Texture Generation
0
Sign in to get full access
Overview
- Garment3DGen is a novel AI system that can generate 3D garments with stylized textures from a given image or sketch.
- The system can produce diverse designs while maintaining semantic consistency with the input.
- It combines 3D garment generation with texture synthesis to create visually appealing and realistic 3D garment models.
Plain English Explanation
The Garment3DGen system allows users to create 3D models of clothing and accessories. By providing an image or sketch as input, the system can generate a 3D garment with a stylized texture.
The key innovation is that Garment3DGen can generate diverse designs that still look consistent with the original input. For example, if you provide a sketch of a t-shirt, the system can produce multiple 3D t-shirt models with different patterns, colors, and details, but they will all clearly be variations of the original t-shirt design.
This combining of 3D garment generation and texture synthesis allows Garment3DGen to create 3D clothing models that are visually appealing and realistic, rather than just generic shapes. The generated garments could be useful for applications like virtual fashion, gaming, and e-commerce.
Technical Explanation
The Garment3DGen system uses a deep learning architecture to generate 3D garment geometry and textures from a given image or sketch. The core of the model is a conditional generative adversarial network (cGAN) that takes in the input image and outputs a 3D garment mesh along with a corresponding UV texture map.
The 3D garment generation component learns to predict the 3D vertex positions and face connectivity of the garment from the input. The texture generation module then synthesizes a high-quality texture map that is consistent with the generated 3D geometry.
Importantly, Garment3DGen is designed to produce diverse outputs while maintaining semantic consistency with the input. This is achieved through the use of a latent code vector that captures the high-level style information, allowing the system to generate multiple variations around a common theme.
The model is trained on a large dataset of 3D garment scans paired with corresponding 2D images. Extensive experiments demonstrate the system's ability to generate visually compelling 3D garments with a wide range of styles and textures.
Critical Analysis
The Garment3DGen paper presents a promising approach for 3D garment generation and texture synthesis. By combining these two key capabilities, the system can create 3D clothing models that are both geometrically accurate and visually appealing.
However, the authors acknowledge some limitations of their work. The current system relies on a dataset of 3D scanned garments, which may not fully capture the diversity of real-world clothing designs. Additionally, the texture synthesis component is limited to 2D textures and does not consider factors like material properties or 3D wrinkle patterns.
Further research could explore ways to expand the system's capabilities, such as generating 3D garments from scratch without relying on a pre-existing dataset, or incorporating more advanced texture synthesis techniques that account for 3D geometry and material characteristics.
It would also be valuable to investigate the system's performance on a wider range of garment types, including more complex and unconventional designs, to better understand its strengths and limitations.
Conclusion
The Garment3DGen system represents an important advancement in the field of 3D garment generation and texture synthesis. By combining these two key capabilities, the system can create visually compelling 3D clothing models that could be useful for a variety of applications, such as virtual fashion, gaming, and e-commerce.
The system's ability to generate diverse designs while maintaining semantic consistency with the input is a particularly noteworthy feature, as it allows for the creation of a wide range of stylized garments from a single source.
While the current version of the system has some limitations, the underlying approach shows great promise and could pave the way for further advancements in this rapidly evolving field of research.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
0
Garment3DGen: 3D Garment Stylization and Texture Generation
Nikolaos Sarafianos, Tuur Stuyck, Xiaoyu Xiang, Yilei Li, Jovan Popovic, Rakesh Ranjan
We introduce Garment3DGen a new method to synthesize 3D garment assets from a base mesh given a single input image as guidance. Our proposed approach allows users to generate 3D textured clothes based on both real and synthetic images, such as those generated by text prompts. The generated assets can be directly draped and simulated on human bodies. We leverage the recent progress of image-to-3D diffusion methods to generate 3D garment geometries. However, since these geometries cannot be utilized directly for downstream tasks, we propose to use them as pseudo ground-truth and set up a mesh deformation optimization procedure that deforms a base template mesh to match the generated 3D target. Carefully designed losses allow the base mesh to freely deform towards the desired target, yet preserve mesh quality and topology such that they can be simulated. Finally, we generate high-fidelity texture maps that are globally and locally consistent and faithfully capture the input guidance, allowing us to render the generated 3D assets. With Garment3DGen users can generate the simulation-ready 3D garment of their choice without the need of artist intervention. We present a plethora of quantitative and qualitative comparisons on various assets and demonstrate that Garment3DGen unlocks key applications ranging from sketch-to-simulated garments or interacting with the garments in VR. Code is publicly available.
Read more8/15/2024
🌿
0
GarmentDreamer: 3DGS Guided Garment Synthesis with Diverse Geometry and Texture Details
Boqian Li, Xuan Li, Ying Jiang, Tianyi Xie, Feng Gao, Huamin Wang, Yin Yang, Chenfanfu Jiang
Traditional 3D garment creation is labor-intensive, involving sketching, modeling, UV mapping, and texturing, which are time-consuming and costly. Recent advances in diffusion-based generative models have enabled new possibilities for 3D garment generation from text prompts, images, and videos. However, existing methods either suffer from inconsistencies among multi-view images or require additional processes to separate cloth from the underlying human model. In this paper, we propose GarmentDreamer, a novel method that leverages 3D Gaussian Splatting (GS) as guidance to generate wearable, simulation-ready 3D garment meshes from text prompts. In contrast to using multi-view images directly predicted by generative models as guidance, our 3DGS guidance ensures consistent optimization in both garment deformation and texture synthesis. Our method introduces a novel garment augmentation module, guided by normal and RGBA information, and employs implicit Neural Texture Fields (NeTF) combined with Score Distillation Sampling (SDS) to generate diverse geometric and texture details. We validate the effectiveness of our approach through comprehensive qualitative and quantitative experiments, showcasing the superior performance of GarmentDreamer over state-of-the-art alternatives. Our project page is available at: https://xuan-li.github.io/GarmentDreamerDemo/.
Read more5/22/2024
0
ClotheDreamer: Text-Guided Garment Generation with 3D Gaussians
Yufei Liu, Junshu Tang, Chu Zheng, Shijie Zhang, Jinkun Hao, Junwei Zhu, Dongjin Huang
High-fidelity 3D garment synthesis from text is desirable yet challenging for digital avatar creation. Recent diffusion-based approaches via Score Distillation Sampling (SDS) have enabled new possibilities but either intricately couple with human body or struggle to reuse. We introduce ClotheDreamer, a 3D Gaussian-based method for generating wearable, production-ready 3D garment assets from text prompts. We propose a novel representation Disentangled Clothe Gaussian Splatting (DCGS) to enable separate optimization. DCGS represents clothed avatar as one Gaussian model but freezes body Gaussian splats. To enhance quality and completeness, we incorporate bidirectional SDS to supervise clothed avatar and garment RGBD renderings respectively with pose conditions and propose a new pruning strategy for loose clothing. Our approach can also support custom clothing templates as input. Benefiting from our design, the synthetic 3D garment can be easily applied to virtual try-on and support physically accurate animation. Extensive experiments showcase our method's superior and competitive performance. Our project page is at https://ggxxii.github.io/clothedreamer.
Read more6/26/2024
0
WordRobe: Text-Guided Generation of Textured 3D Garments
Astitva Srivastava, Pranav Manu, Amit Raj, Varun Jampani, Avinash Sharma
In this paper, we tackle a new and challenging problem of text-driven generation of 3D garments with high-quality textures. We propose WordRobe, a novel framework for the generation of unposed & textured 3D garment meshes from user-friendly text prompts. We achieve this by first learning a latent representation of 3D garments using a novel coarse-to-fine training strategy and a loss for latent disentanglement, promoting better latent interpolation. Subsequently, we align the garment latent space to the CLIP embedding space in a weakly supervised manner, enabling text-driven 3D garment generation and editing. For appearance modeling, we leverage the zero-shot generation capability of ControlNet to synthesize view-consistent texture maps in a single feed-forward inference step, thereby drastically decreasing the generation time as compared to existing methods. We demonstrate superior performance over current SOTAs for learning 3D garment latent space, garment interpolation, and text-driven texture synthesis, supported by quantitative evaluation and qualitative user study. The unposed 3D garment meshes generated using WordRobe can be directly fed to standard cloth simulation & animation pipelines without any post-processing.
Read more7/16/2024