ComfyUI Extension: Comfyui-SAL-VTON

Comfyui-SAL-VTON Introduction

Comfyui-SAL-VTON is an extension designed to help AI artists dress up their models virtually. This tool allows you to apply garments to images of people, creating a virtual try-on experience. It is based on the research paper Linking Garment with Person via Semantically Associated Landmarks for Virtual Try-On (https://openaccess.thecvf.com/content/CVPR2023/papers/Yan_Linking_Garment_With_Person_via_Semantically_Associated_Landmarks_for_Virtual_CVPR_2023_paper.pdf) by Keyu Y. Tingwei G. et al. (2023). The extension leverages advanced AI models to seamlessly integrate clothing onto a person’s image, making it a valuable tool for fashion designers, digital artists, and anyone interested in virtual fashion.

How Comfyui-SAL-VTON Works

Comfyui-SAL-VTON works by using AI models to identify key landmarks on both the garment and the person. These landmarks help the system understand how the garment should be warped and fitted onto the person. Think of it like a digital tailor that knows exactly where to place each part of the clothing to make it look natural on the person.

Here’s a simplified breakdown of the process:

1. Landmark Detection: The system first identifies important points on the garment and the person’s image.
2. Warping: The garment is then digitally warped to match the shape and pose of the person.
3. Overlay: Finally, the warped garment is overlaid onto the person’s image, creating a realistic virtual try-on effect.

Comfyui-SAL-VTON Features

Landmark Detection

This feature identifies key points on the garment and the person’s image. These points are crucial for ensuring that the garment fits naturally on the person.

Warping

The warping feature adjusts the garment to match the person’s pose and body shape. This ensures that the clothing looks as if it is naturally draped over the person.

Overlay

The overlay feature combines the warped garment with the person’s image, creating a seamless and realistic virtual try-on experience.

Customization

You can customize the input images to ensure the best results. For instance, the garment image should be 768x1024 pixels. If your image is not of this size, you can resize it using ComfyUI. Additionally, for better results, the background of the person’s image should be white. You can use tools like rembg or SAM to mask the background and replace it with white.

Comfyui-SAL-VTON Models

Comfyui-SAL-VTON uses three main models:

1. Landmark Model: This model detects key landmarks on the garment and the person.

• Download Landmark Model

2. Warp Model: This model warps the garment to fit the person’s shape and pose.

• Download Warp Model

3. SAL-VTON Model: This is the main model that integrates the garment with the person’s image.

• Download SAL-VTON Model

Troubleshooting Comfyui-SAL-VTON

Common Issues and Solutions

1. Poor Results with Non-White Backgrounds:

• Solution: Use tools like rembg or SAM to mask the background and replace it with white.

2. Incorrect Garment Size:

• Solution: Ensure the garment image is 768x1024 pixels. Resize the image in ComfyUI if necessary.

3. Garment Mask Issues:

• Solution: The garment mask should match the shape of the input garment. You can generate it using SAM or rembg.

Frequently Asked Questions

Q: What should I do if the garment doesn’t fit well on the person’s image? A: Make sure the garment image is the correct size (768x1024 pixels) and the person’s image has a white background.

Q: Can I use any image for the person? A: Yes, but for the best results, the person’s image should have a white background.

Learn More about Comfyui-SAL-VTON

For additional resources, tutorials, and community support, you can explore the following:

• ModelScope GitHub Repository
• ModelScope Documentation
• ModelScope Community Forums These resources provide detailed guides, examples, and a platform to ask questions and get support from other users and developers.