LivePortrait | Animate Portraits | Img2Vid

What is LivePortrait?

LivePortrait is an efficient portrait animation framework developed by Kuaishou Technology that aims to synthesize lifelike videos from a single source image. It uses the source image as an appearance reference and derives motion (facial expressions and head pose) from a driving video, audio, text, or generation.

Unlike mainstream diffusion-based methods, LivePortrait explores and extends an implicit-keypoint-based framework to effectively balance computational efficiency and controllability. It focuses on better generalization, controllability, and efficiency for practical usage. LivePortrait has high generation quality even compared to diffusion methods, while being extremely fast - about 12.8ms per frame on an RTX 4090 GPU with PyTorch.

For more details, visit

How LivePortrait Works

At a high level, LivePortrait takes a source image and a driving video as inputs. It extracts the appearance from the source image and motion from the driving video. These are then combined using warping and generation modules to synthesize an animated portrait video that retains the identity of the source image but follows the motion and expressions of the driving video.

The key components in LivePortrait are:

1. Appearance Feature Extractor: Encodes the identity and appearance information from the source image.
2. Motion Extractor: Extracts motion features (facial keypoints) from the driving video frames.
3. Warping Module: Uses the extracted motion to warp the source image features, aligning them to the driving pose and expression.
4. Image Generator: Takes the warped features and synthesizes the final photorealistic animated frame.
5. Stitching & Retargeting Module: Optionally stitches the generated portrait back onto the original image and allows controlling specific facial regions like eyes and mouth.

These modules are efficiently designed and work together to enable high-quality, controllable LivePortrait animation.

How to Use ComfyUI LivePortrait

Thanks to  node and workflow, creating realistic portrait animations in ComfyUI is now easier. The following is a breakdown of the key components and parameters of his ComfyUI LivePortrait workflow.

Key Steps of ComfyUI LivePortrait Img2Vid Workflow

1. Load the Live Portrait Models

• Add the "DownloadAndLoadLivePortraitModels" node
• Set precision to auto or fp16 for best performance

2. Choose Face Detector for LivePortrait

• You have a choice between the "LivePortraitLoadCropper" (InsightFace) and "LivePortraitLoadMediaPipeCropper" nodes
• InsightFace is more accurate but has a non-commercial license, while MediaPipe is faster on CPU but less accurate
• Both output a "cropper" that will be used to detect and crop faces

3. Load and Preprocess Source Image for LivePortrait

• Load your source portrait image using the "Load Image" node
• Resize it to 512x512 using the "ImageResize" node
• Connect the resized image to a "LivePortraitCropper" node
• Also connect your selected face detector's "cropper" output to this node
• Key parameters in "LivePortraitCropper" Node

"dsize": This sets the output resolution of the cropped face image

• Default is 512, meaning the face will be cropped to 512x512 pixels
• Higher values will crop the face in higher resolution, but may be slower to process
• Lower values will be faster but may lose some detail

"scale": This controls how zoomed in the face crop will be

• Default is 2.3, higher values will zoom in closer on the face, lower values will include more of the head/background
• You want to adjust this so the crop includes the entire face and some background, but not too much extra space
• A good face crop is important for the motion transfer to work well
• Typical values range from 1.8 to 2.5 depending on the framing of the source image

"face_index": If there are multiple faces detected in the image, this selects which one to crop

• Default is 0, which selects the first detected face
• Increase this if you want to select a different face in the image
• Detected faces are ordered based on the "face_index_order" setting (default is largest to smallest)

"vx_ratio" and "vy_ratio" (Optional): These allow you to offset the crop vertically (vy) or horizontally (vx)

• Values range from -1 to 1
• For example, setting vy to 0.1 will shift the crop upwards by 10% of the frame size
• This can help if the automatic crop is slightly misaligned

"face_index_order": This sets how detected faces are ordered when selecting with face_index

• Default is "large-small" which orders from largest to smallest face
• Can also order from left to right, top to bottom, etc.
• This is only relevant if you have multiple faces in your image

4. Load and Preprocess Driving Video for LivePortrait

• Load your driving video using the "VHS_LoadVideo" node
• Adjust num_frames using the "frame_load_cap" primitive
• Resize the video frames to 480x480 using a "GetImageSizeAndCount" node
• You can optionally crop the driving video frames using another "LivePortraitCropper" node

5. Apply Motion Transfer for LivePortrait

• Add the "LivePortraitProcess" node
• Connect the loaded pipeline, source image crop_info, cropped source image, and driving frames to "LivePortraitProcess" node
• Key parameters in "LivePortraitProcess" Node

"lip_zero": When enabled, this will zero out the lip parameters if they fall below a certain threshold

• This can help reduce unnatural lip movements and improve lip sync
• Recommended to enable this unless you specifically want to preserve all lip motion

"lip_zero_threshold": This sets the threshold below which lip parameters will be zeroed out when "lip_zero" is enabled

• Default is 0.03, higher values will zero out more lip motion, lower values will preserve more
• Adjust this if you want to change how much lip motion is suppressed

"stitching": When enabled, this will blend the animated face back into the original image using a stitching process

• This can help create a more seamless transition between the animated face and the background
• Recommended to enable this for the most natural-looking results

"delta_multiplier": This scales the motion parameters by a multiplier

• Default is 1.0, higher values will exaggerate motion, lower values will reduce it
• Can be used to adjust the overall intensity of the facial motion
• Typical values range from 0.8 to 1.5 depending on the desired effect

"mismatch_method": This sets how the workflow handles mismatches between the number of source and driving frames

• Options are "constant", "cycle", "mirror", and "cut"
• "constant" will hold on the last frame, "cycle" will loop, "mirror" will play forwards then backwards, "cut" will stop
• Default is "constant", change this if you want a different behavior when the driving video is longer or shorter than the source

"relative_motion_mode": This controls how motion is transferred from the driving video to the source image

• Options are "relative", "source_video_smoothed", "relative_rotation_only", "single_frame", and "off"
• "relative" is the default and uses relative motion transfer
• "off" will disable motion transfer entirely
• Experiment with different modes to see which gives the best results for your specific use case

"driving_smooth_observation_variance": This controls the smoothness of the driving motion when using the "source_video_smoothed" motion mode

• Higher values will smooth out the motion more, lower values will preserve more of the original motion
• Default is 0.000003, adjust this if you want to change the smoothness of the transferred motion

6. Composite Result (Optional) for LivePortrait

• To composite the animated face back into the source image, use the "LivePortraitComposite" node
• Connect the original source image, cropped animated image, LivePortrait output data, and an optional mask
• This will output full frames with the animated face blended in

7. Configure Retargeting (Optional) for LivePortrait

• For finer control over eyes and lips, use the "LivePortraitRetargeting" node
• Enable eye and/or lip retargeting and adjust their multipliers
• Connect the retargeting info to "LivePortraitProcess"

Please note that the Insightface model is required in this workflow. Insightface model (https://github.com/deepinsight/insightface/releases/download/v0.7/buffalo_l.zip) license is non-commercial in nature.

If you are interested in LivePortrait Vid2Vid, please use