ComfyUI Extension: ComfyUI-InstantID

ComfyUI-InstantID Introduction

ComfyUI-InstantID is an unofficial implementation of the InstantID project, designed to work within the ComfyUI framework. This extension allows AI artists to generate identity-preserving images using a single reference image. It supports various styles and poses, making it a versatile tool for creating unique and consistent artwork. Whether you're looking to maintain the identity of a subject across different artistic styles or need to generate images with specific poses, ComfyUI-InstantID can help streamline your creative process.

How ComfyUI-InstantID Works

ComfyUI-InstantID leverages advanced AI models to generate images that preserve the identity of a subject based on a reference image. It uses a combination of base models, ControlNet models, and IPAdapter models to achieve this. The process involves loading a reference image, applying various models to extract and maintain identity features, and then generating new images that retain these features while allowing for stylistic and pose variations.

Think of it as a sophisticated photo editor that not only changes the style of an image but also ensures that the subject's identity remains recognizable. This is particularly useful for artists who need to create multiple versions of a character or subject without losing their unique features.

ComfyUI-InstantID Features

Base Model Loader

• ID Base Model Loader from hub: Automatically downloads models from the Huggingface hub by entering the model name (e.g., wangqixun/YamerMIX_v8).
• ID Base Model Loader locally: Supports loading local models, specifically from the SDXL series.

InsightFace Loader

• Supports both CUDA and CPU, allowing for flexible hardware usage.

ID ControlNet Loader

• controlnet_path: Specifies the path to the ID ControlNet model.

Ipadapter_instantid Loader

• Ipadapter_instantid_path: Path to the IPAdapter model.
• filename: Name of the model file.

ID Prompt_Styler

• Compatible with various text prompts and stylers, including Photomaker Prompt_Styler.
• prompt, negative: Positive and negative prompts.
• style_name: Supports eight official styles:
• (No style)
• Watercolor
• Film Noir
• Neon
• Jungle
• Mars
• Vibrant Color
• Snow
• Line art

InstantID Generation

• face_image: Input for the reference face image.
• pipe: Connects to the model pipeline.
• insightface: Connects to the InsightFace model.
• pose_image_optional: Optional input for a pose reference image (affects only the area around the face).
• positivet, negative: Positive and negative prompts.
• ip_adapter_scale: Strength of the IPAdapter.
• controlnet_conditioning_scale: Strength of the ID ControlNet.
• step: Number of steps, with a default of 30.
• guidance_scale: Prompt relevance, usually set to 5.
• enhance_face_region: Option to enhance the face region.
• seed: Seed for random number generation.

ComfyUI-InstantID Models

ComfyUI-InstantID supports various models that can be loaded either from the Huggingface hub or locally. These models include base models, ControlNet models, and IPAdapter models. Each model serves a specific purpose, such as maintaining identity features, applying stylistic changes, or conditioning the image generation process.

Example Models

• Base Models: Used for general image generation.
• ControlNet Models: Provide additional control over the generated images.
• IPAdapter Models: Enhance the identity-preserving capabilities of the extension.

What's New with ComfyUI-InstantID

Version 2.0

• Added support for pose reference images.
• Optimized code for better performance.
• Fixed conflicts with the InsightFace loader.
• Improved compatibility with different versions of onnxruntime-gpu.

Version 1.0

• Initial release with support for local and Huggingface hub models.
• Included eight different styles for image generation.

Troubleshooting ComfyUI-InstantID

Common Issues and Solutions

1. Model Loading Errors:

• Ensure that the model paths are correctly specified.
• Verify that the models are downloaded and placed in the correct directories.

2. Performance Issues:

• Check if your hardware supports CUDA for faster processing.
• Adjust the number of steps and scales to optimize performance.

3. Image Quality Problems:

• Experiment with different prompts and styles.
• Use the enhance_face_region option to improve facial features.

Frequently Asked Questions

• Q: Can I use my own models?
• A: Yes, you can load local models by specifying the correct paths.
• Q: How do I improve the likeness of the generated images?
• A: Increase the ip_adapter_scale and controlnet_conditioning_scale values.

Learn More about ComfyUI-InstantID

For additional resources, tutorials, and community support, you can explore the following links:

• InstantID GitHub Repository
• ComfyUI Documentation
• Huggingface Models These resources provide comprehensive guides, example workflows, and community forums where you can ask questions and share your experiences.