This paper introduces a novel approach for generating personalized 3D avatars for virtual try-on applications. Utilizing Gaussian Splatting for 3D reconstruction from five-perspective images, combined with advanced 3D segmentation techniques and photogrammetry-based clothing digitization, this method addresses the limitations of existing systems in personalization and garment fitting accuracy. The proposed pipeline integrates state-of-the-art tools such as Blender addons to deliver a seamless virtual try-on experience. Experimental results demonstrate significant improvements in avatar realism and garment fit, paving the way for enhanced e-commerce solutions.
The rise of e-commerce has underscored the need for realistic and accurate virtual try-on systems. Current solutions often fall short in personalization, relying heavily on generalized 3D models that fail to capture unique body features. Furthermore, achieving a natural fit between garments and avatars remains a persistent challenge.
This work proposes a new method that integrates Gaussian Splatting for personalized 3D avatar generation, 3D segmentation to define body regions, and photogrammetry for realistic garment modeling. The result is a robust pipeline capable of delivering highly accurate and visually appealing virtual try-on experiences.
Certainly, here are the links to the related works mentioned in your publication:
Image Collection: Five-perspective images (front, back, left, right, top) are captured to ensure complete coverage of the subject.
Garment Scanning: Using photogrammetry, garments are digitized to create high-resolution 3D models. The workflow involves structured lighting setups and processing tools such as Meshroom or Agisoft Metashape.
Gaussian Splatting: Gaussian Splatting is employed to reconstruct a 3D model from the captured images. This involves mapping the image data into a 3D space, representing it as a collection of Gaussian distributions for efficient rendering and modeling.
Body Part Segmentation: A segmentation algorithm is applied to group vertices into predefined regions (e.g., head, neck, torso, arms, legs). This segmentation ensures accurate garment fitting and improves the realism of the virtual try-on experience.
The proposed pipeline consists of the following steps:
Hardware requirements include a high-resolution camera, a photogrammetry setup, and a computing system with GPU support for 3D modeling. Software tools such as Blender, Meshroom, and custom Python scripts facilitate the implementation.
The integration of Gaussian Splatting and photogrammetry represents a significant advancement in virtual try-on technology. By addressing the limitations of existing methods, this approach offers enhanced personalization and accuracy. However, challenges remain in processing time and scalability, which can be mitigated with further optimization.
This paper presents a comprehensive framework for creating personalized 3D avatars for virtual try-on applications. By leveraging Gaussian Splatting, 3D segmentation, and photogrammetry, the proposed method achieves high levels of realism and fit accuracy. Future work will explore real-time processing and dynamic garment simulation to further enhance user experience.
[1]. Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., & Black, M. J. (2015). SMPL: A Skinned Multi-Person Linear Model. ACM Trans. Graphics (Proc. SIGGRAPH Asia), 34(6), 248:1--248:16.
[2]. Pavlakos, G., Choutas, V., Ghorbani, N., Bolkart, T., Osman, A. A. A., Tzionas, D., & Black, M. J. (2019). Expressive Body Capture: 3D Hands, Face, and Body from a Single Image. Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 10975-10985.
[3]. Xu, X., Liu, L., & Yan, S. (2024). SMPLer: Taming Transformers for Monocular 3D Human Shape and Pose Estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46, 3275-3289.
[4]. Zhu, H., Cao, Y., Jin, H., Chen, W., Du, D., Wang, Z., Cui, S., & Han, X. (2020). Deep Fashion3D: A Dataset and Benchmark for 3D Garment Reconstruction from Single Images. arXiv:2003.12753 [cs.CV].