ComfyUI Extension: ComfyUI-Ruyi

Ruyi-Models Introduction

Welcome to Ruyi-Models! This extension is designed to transform your static images into cinematic-quality videos. Ruyi-Models can generate videos with a resolution of 768 pixels, at 24 frames per second, for a total of 5 seconds (120 frames). It offers features like camera control and motion amplitude control, allowing you to create dynamic and engaging video content. With the power of RTX 3090 or RTX 4090 graphics cards, you can produce high-quality videos without any loss in precision, even at 512 resolution for 120 frames or 768 resolution for approximately 72 frames. This tool is perfect for AI artists looking to bring their creations to life with stunning video outputs.

How Ruyi-Models Works

Ruyi-Models operates by taking an input image and transforming it into a video sequence. Think of it as a digital flipbook where each frame is a slightly altered version of the previous one, creating the illusion of motion. The extension uses advanced algorithms to ensure smooth transitions and realistic motion, making your static images come alive. By adjusting parameters like camera angles and motion amplitude, you can control how your video evolves, much like directing a scene in a movie.

Ruyi-Models Features

• High-Resolution Video Generation: Create videos at 768 resolution with 24 frames per second, ensuring high-quality output.
• Camera Control: Adjust the camera's movement to pan left, right, up, or down, or keep it static, giving you creative control over the video perspective.
• Motion Amplitude Control: Fine-tune the intensity of motion within your video, from subtle movements to more dynamic actions.
• Optimized for High-End GPUs: Leverage the power of RTX 3090 and RTX 4090 for efficient video generation without compromising on quality.

Ruyi-Models Models

Ruyi-Models offers different models tailored to various needs. For instance, the Ruyi-Mini-7B model is designed for generating videos at 512 and 768 resolutions, with a maximum of 120 frames at 24 frames per second. This model is ideal for artists who need high-quality video outputs without extensive computational resources.

What's New with Ruyi-Models

• January 14, 2025: Integration of TeaCache and Enhance-A-Video as nodes in ComfyUI, offering faster generation speeds and improved video quality.
• January 6, 2025: Added FP8 support to ComfyUI with three modes: bf16 default, fp8 lite, fp8 strong, and fp8 extreme, optimizing memory usage.
• December 24, 2024: Updated transformer models to fix black bar issues in 3:4 and 4:5 aspect ratio videos.

Troubleshooting Ruyi-Models

If you encounter issues while using Ruyi-Models, here are some common solutions:

• Video Quality Issues: Ensure your GPU drivers are up to date and that you are using the recommended hardware (RTX 3090 or RTX 4090) for optimal performance.
• Installation Problems: Verify that all dependencies are correctly installed. Refer to the installation section for detailed steps.
• Performance Bottlenecks: Adjust the GPU_memory_mode and GPU_offload_steps settings to optimize memory usage and improve processing times.

Learn More about Ruyi-Models

To further explore Ruyi-Models, consider visiting the following resources:

• ComfyUI-Manager: A tool to manage and enhance your ComfyUI experience.
• Enhance-A-Video Blog (https://oahzxl.github.io/Enhance_A_Video/): Learn more about video enhancement techniques.
• TeaCache Project Page: Discover how caching can accelerate video diffusion models. These resources provide valuable insights and community support to help you make the most of Ruyi-Models.