ComfyUI  >  Nodes  >  segment anything

ComfyUI Extension: segment anything

Repo Name

comfyui_segment_anything

Author
storyicon (Account age: 2561 days)
Nodes
View all nodes (5)
Latest Updated
7/12/2024
Github Stars
0.6K

How to Install segment anything

Install this extension via the ComfyUI Manager by searching for  segment anything
  • 1. Click the Manager button in the main menu
  • 2. Select Custom Nodes Manager button
  • 3. Enter segment anything in the search bar
After installation, click the  Restart button to restart ComfyUI. Then, manually refresh your browser to clear the cache and access the updated list of nodes.

Visit ComfyUI Online for ready-to-use ComfyUI environment

  • Free trial available
  • High-speed GPU machines
  • 200+ preloaded models/nodes
  • Freedom to upload custom models/nodes
  • 50+ ready-to-run workflows
  • 100% private workspace with up to 200GB storage
  • Dedicated Support

Run ComfyUI Online

segment anything Description

ComfyUI Extension's 'segment anything' leverages GroundingDino and SAM to use semantic strings for segmenting any element within an image, offering a specialized version of sd-webui-segment-anything.

segment anything Introduction

The comfyui_segment_anything extension is a powerful tool designed to integrate advanced image segmentation capabilities into the ComfyUI environment. This extension leverages state-of-the-art models to enable precise and efficient segmentation of images, which can be particularly useful for AI artists looking to enhance their creative workflows. By using this extension, you can easily generate segmentation masks, automate image matting, and improve inpainting tasks, making it a versatile addition to your AI art toolkit.

How segment anything Works

At its core, comfyui_segment_anything uses advanced machine learning models to analyze and segment images. Segmentation is the process of partitioning an image into multiple segments or regions, often to isolate specific objects or areas of interest. This extension simplifies this complex task by providing an intuitive interface and automated processes.

Imagine you have a photograph with various objects, and you want to isolate a particular object, like a tree. The extension uses models like SAM (Segment Anything Model) and GroundingDINO to identify and create a mask around the tree, separating it from the rest of the image. This mask can then be used for various purposes, such as inpainting, where you can fill in or modify the background while keeping the tree intact.

segment anything Features

Core Features

  1. Segmentation Mask Generation: Automatically generate masks for different objects in an image. This is useful for isolating parts of an image for further editing or analysis.
  2. Inpainting Support: Enhance inpainting tasks by using generated masks to fill in or modify specific areas of an image.
  3. Image Matting: Automate the process of extracting objects from their backgrounds, which is particularly useful for creating clean and professional-looking images.
  4. Integration with ControlNet: Improve semantic segmentation and inpainting by integrating with ControlNet, allowing for more precise and detailed edits.

Customization Options

  • Point Prompts: Add positive or negative points on the image to guide the segmentation process.
  • Text Prompts: Use text descriptions to generate bounding boxes around objects, which can then be converted into masks.
  • Mask Expansion: Adjust the size of the generated masks to better fit the objects of interest.

segment anything Models

The extension supports several models, each suited for different tasks and performance requirements:

bert-base-uncased

  • Purpose: Used for text-based segmentation tasks.
  • Usage: Automatically downloaded during inference or can be manually downloaded from .

GroundingDino

  • Purpose: Generates bounding boxes and segmentation masks based on text prompts.
  • Models:
  • GroundingDINO_SwinT_OGC: ,
  • GroundingDINO_SwinB: ,

SAM (Segment Anything Model)

  • Purpose: General-purpose segmentation.
  • Models:
  • sam_vit_h: Download (https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth)
  • sam_vit_l: Download (https://dl.fbaipublicfiles.com/segment_anything/sam_vit_l_0b3195.pth)
  • sam_vit_b: Download (https://dl.fbaipublicfiles.com/segment_anything/sam_vit_b_01ec64.pth)
  • sam_hq_vit_h:
  • sam_hq_vit_l:
  • sam_hq_vit_b:
  • mobile_sam:

What's New with segment anything

Recent Updates

  • v1.0.0: Initial release with core functionalities.
  • v1.0.1: Added mask expansion and API support.
  • v1.1.0: Introduced GroundingDINO support for text-based segmentation.
  • v1.2.0: Added ControlNet inpainting support.
  • v1.3.0: Automatic segmentation support for various tasks.
  • v1.4.0: Complete API refactor for single image processing.
  • v1.4.1: EditAnything integration for random segmentation.
  • v1.4.2: CPU support for SAM inference.
  • v1.5.0: Local GroundingDINO support to bypass C++ issues.
  • v1.5.1: Reintroduced "Upload Mask to ControlNet Inpainting" feature.
  • v1.6.0: Added support for SAM-HQ.
  • v1.6.1: Added support for MobileSAM.
  • v1.6.2: Compatibility with WebUI v1.6.0 and Gradio v3.41.2. ## Troubleshooting segment anything

Common Issues and Solutions

  1. Slow Model Downloads:
  • Solution: Set the HTTP_PROXY and HTTPS_PROXY environment variables to use a proxy.
  1. GroundingDINO Installation Problems:
  • Solution: Use the local GroundingDINO option to bypass C++/CUDA issues. Refer to the for detailed steps.
  1. ControlNet Compatibility Issues:
  • Solution: Ensure that the SAM extension is loaded before ControlNet by adjusting the extension order.

Frequently Asked Questions

  • Why is my segmentation mask not accurate?
  • Ensure that you are using the correct model and settings for your specific task. Adjust point prompts and mask expansion as needed.
  • Can I use this extension on a CPU?
  • Yes, but it will be significantly slower than using a GPU. Enable the "Use CPU for SAM" option if necessary.

Learn More about segment anything

For additional resources, tutorials, and community support, consider the following:

  • : Detailed guides and API documentation.
  • : Engage with other users and developers.
  • : Step-by-step video guides (in Chinese). By leveraging these resources, you can maximize the potential of the comfyui_segment_anything extension and enhance your AI art projects.

segment anything Related Nodes

RunComfy

© Copyright 2024 RunComfy. All Rights Reserved.

RunComfy is the premier ComfyUI platform, offering ComfyUI online environment and services, along with ComfyUI workflows featuring stunning visuals.