ComfyUI Extension: ComfyUI-AdvancedLivePortrait

ComfyUI-AdvancedLivePortrait Introduction

ComfyUI-AdvancedLivePortrait is an advanced extension designed to enhance your AI art projects by allowing you to edit and animate facial expressions in photos and videos. This extension provides a faster and more efficient way to create real-time previews, making it easier for you to see the results of your edits instantly. Whether you want to add expressions to a static image, insert them into a video, or create animations using multiple facial expressions, ComfyUI-AdvancedLivePortrait has got you covered.

ComfyUI-AdvancedLivePortrait

Key Features:

• Edit Facial Expressions: Modify the expressions in your photos to convey different emotions.
• Insert Expressions into Videos: Seamlessly add facial expressions to video footage.
• Create Animations: Generate animations by combining multiple facial expressions.
• Extract Expressions: Extract facial expressions from sample photos for reuse.

How ComfyUI-AdvancedLivePortrait Works

ComfyUI-AdvancedLivePortrait operates by using advanced facial recognition and animation techniques to manipulate facial expressions in both images and videos. Here’s a simplified breakdown of how it works:

1. Facial Detection: The extension uses facial detection models to identify and map facial features in your images or videos.
2. Expression Mapping: Once the facial features are detected, the extension allows you to map new expressions onto the detected faces.
3. Real-Time Preview: You can see the changes in real-time, which helps in making quick adjustments and achieving the desired look.
4. Animation Creation: By combining different expressions, you can create smooth animations that bring your characters to life.

ComfyUI-AdvancedLivePortrait Features

Edit Facial Expressions

You can easily modify the facial expressions in your photos. This feature is perfect for creating different emotional states for your characters. For example, you can turn a neutral face into a smiling one or add a surprised look.

Insert Expressions into Videos

This feature allows you to add facial expressions to video footage. By linking the driving video to 'src_images', you can animate the faces in your videos with new expressions, making your video content more dynamic and engaging.

Create Animations

With the ability to combine multiple facial expressions, you can create animations that show a range of emotions. This is particularly useful for creating animated sequences or enhancing storytelling in your projects.

Extract Facial Expressions

You can extract facial expressions from sample photos and save them for later use. This feature is handy for building a library of expressions that you can quickly apply to different projects.

Save and Load Expressions

The extension allows you to save and load expressions using the 'Load Exp Data' and 'Save Exp Data' nodes. This makes it easy to reuse expressions across different projects.

ComfyUI-AdvancedLivePortrait Models

ComfyUI-AdvancedLivePortrait supports different facial detection models, each with its own strengths:

Insightface

• Usage: Best for high-accuracy facial detection.
• License: Non-commercial use only.
• Installation: Requires manual installation. You can install it using pip install insightface or by following the instructions for the portable version.

MediaPipe

• Usage: Suitable for faster detection on CPU, especially on Windows.
• License: Covered under MIT and Apache-2.0 licenses.
• Advantages: Faster on CPU compared to Insightface, but cannot run on GPU in Windows.

Face Alignment

• Usage: Ideal for detecting smaller faces.
• Advantages: Uses the blazeface back camera model, which is better for smaller faces than MediaPipe.

What's New with ComfyUI-AdvancedLivePortrait

Recent Updates

• New Face Detector: Added an alternative face detector from the face-alignment project, which is better for smaller faces.
• Speed and Efficiency: Improved speed and efficiency, allowing for near real-time previews with minimal delay.
• Vid2Vid Smoothing Algorithm: Enhanced video-to-video facial expression transfer with a smoothing algorithm.
• Restructured Nodes: More options and better organization of nodes for easier use.
• Auto Frame Skipping: Automatically skips frames with no face detected, improving processing speed.
• Mac Support: Better support for Mac users on MPS.

Troubleshooting ComfyUI-AdvancedLivePortrait

Common Issues and Solutions

Issue: Slow Initial Run

• Solution: The first run may take longer due to model warmup. Subsequent runs will be faster.

Issue: Insightface Installation Fails

• Solution: Ensure you have a working compile environment. If issues persist, refer to the troubleshooting section of the reactor node here.

Issue: No Face Detected

• Solution: Ensure the face is clearly visible and well-lit. Try using the face-alignment model for better detection of smaller faces.

Frequently Asked Questions

Q: Can I use this extension for commercial projects? A: If you are using the Insightface model, it is strictly for non-commercial use. For commercial projects, consider using MediaPipe.

Q: How do I save and load expressions? A: Use the 'Load Exp Data' and 'Save Exp Data' nodes. The data is saved in the \ComfyUI\output\exp_data\ folder.

Learn More about ComfyUI-AdvancedLivePortrait

For more information, tutorials, and community support, check out the following resources:

• Original Author's Link: LivePortrait
• Model Conversion by Kijai: ComfyUI-LivePortraitKJ
• Community Forums: Join discussions and get support from other AI artists. By leveraging these resources, you can master the use of ComfyUI-AdvancedLivePortrait and take your AI art projects to the next level.