ComfyUI Extension: ComfyUI-Flowty-CRM

ComfyUI-Flowty-CRM Introduction

ComfyUI-Flowty-CRM is an extension for ComfyUI that allows you to use Convolutional Reconstruction Models (CRM) directly within the ComfyUI environment. CRM is a high-fidelity, feed-forward model that can generate 3D textured meshes from a single image in just 10 seconds. This extension integrates CRM into ComfyUI, making it easier to use and more accessible for AI artists.

By using ComfyUI-Flowty-CRM, you can transform 2D images into detailed 3D models, which can be particularly useful for artists looking to create 3D assets from their existing 2D artwork. This extension simplifies the process, allowing you to focus on your creative work without worrying about the technical complexities of 3D model generation.

How ComfyUI-Flowty-CRM Works

ComfyUI-Flowty-CRM works by leveraging the power of Convolutional Reconstruction Models to convert 2D images into 3D textured meshes. Here’s a simplified breakdown of how it works:

1. Input Image Processing: The extension automatically processes the input image to prepare it for 3D reconstruction.
2. Inference Pipeline: The inference pipeline is divided into three parts, allowing it to run on low VRAM GPUs and giving users the flexibility to alter inputs at any step.
3. 3D Model Generation: The processed image is fed into the CRM, which generates a high-quality 3D model with textures. The output can be in GLB or OBJ formats. This process is designed to be user-friendly, so you don’t need to have a deep technical understanding to create impressive 3D models from your 2D images.

ComfyUI-Flowty-CRM Features

ComfyUI-Flowty-CRM comes with several features that make it a powerful tool for AI artists:

• CPU and GPU Support: Initially, CRM could only run on CUDA, but this extension adds support for CPU generation, making it accessible to more users.
• No Xformers Required: The extension removes the need for xformers, simplifying the setup process.
• Flexible Inference Pipeline: The pipeline is separated into three parts, allowing it to run on low VRAM GPUs and giving users the freedom to modify inputs at any stage.
• Automatic Input Image Processing: The extension automatically processes input images, saving you time and effort.
• High-Speed and High-Quality Outputs: Generate high-quality 3D models quickly, with outputs available in GLB and OBJ formats.
• Integration with ComfyUI: The extension is closely integrated with ComfyUI’s internal attention code, ensuring smooth operation and compatibility.
• Optimized Code: The code is cleaner, with fewer dependencies and reduced memory usage, making it more efficient.

ComfyUI-Flowty-CRM Models

The extension uses the CRM model to generate 3D textured meshes. Here’s a brief overview of the model:

• CRM (Convolutional Reconstruction Model): This model is designed to generate high-fidelity 3D textured meshes from a single image. It works in three stages:

1. View Generation: Generates six different views (front, back, left, right, top, and down) from the input image.
2. CCM Generation: Creates Canonical Coordinate Maps (CCMs) for each view.
3. 3D Mesh Generation: Combines the views and CCMs to produce a detailed 3D mesh. You can download the CRM models from Hugging Face and place them in the ComfyUI/models/checkpoints directory.

Troubleshooting ComfyUI-Flowty-CRM

Here are some common issues you might encounter while using ComfyUI-Flowty-CRM and their solutions:

• NVdiffrast Issues:
• Windows Users: If you have a CUDA (NVIDIA) GPU, you might need special preparations to make the nvdiffrast module work. Check the official documentation for installation instructions.
• CUDA_HOME Environment Variable: If you see an error stating CUDA_HOME environment variable is not set, you need to set an environment variable pointing to your CUDA installation path. Refer to this StackOverflow answer for guidance.
• Too Many Issues with NVdiffrast: If you encounter too many issues, consider using the CPU workflow by restarting ComfyUI with the cpu-only option (note that this will be slower).
• Device Mismatch Error:
• Error Message: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
• Solution: This usually happens if you tried to run the CPU workflow but have a CUDA GPU. Restart ComfyUI and run only the CUDA workflow.
• Portable ComfyUI:
• Dependencies: Users might need to install dependencies differently. Refer to this issue comment for more information.
• Insufficient VRAM/RAM:
• Minimum Requirements: You should have at least 8GB of VRAM and 16GB of RAM to run CRM.
• Low VRAM GPUs: For GPUs with less than 16GB of VRAM, you might need to split the workflow into four parts and run them separately. Example workflows for low VRAM setups are available here.

Learn More about ComfyUI-Flowty-CRM

To learn more about ComfyUI-Flowty-CRM and get additional support, check out the following resources:

• CRM Project Page (https://ml.cs.tsinghua.edu.cn/~zhengyi/CRM/): Detailed information about the CRM model.
• Official CRM Implementation: The original implementation of the CRM model.
• Hugging Face Demo: Try out the CRM model online.
• ComfyUI Essentials: Additional nodes and features for ComfyUI.
• Community Forums: Ask questions and get support from the community. By exploring these resources, you can deepen your understanding of ComfyUI-Flowty-CRM and make the most of its capabilities in your creative projects.