FLUX IP-Adapter | ImageGen with Style Adaptation

FLUX is a new image generation model developed by . The FLUX-IP-Adapter checkpoint for the FLUX.1-dev model is provided by the XLabs AI team, who also created the ComfyUI FLUX-IP-Adapter workflow. For more details, please visit . All credit goes to their contribution.

About FLUX

The FLUX models are preloaded on RunComfy, named flux/flux-schnell and flux/flux-dev.

• When launch a RunComfy Medium-Sized Machine: Select the checkpoint flux-schnell, fp8 and clip t5_xxl_fp8 to avoid out-of-memory issues.
• When launch a RunComfy Large-Sized or Above Machine: Opt for a large checkpoint flux-dev, default and a high clip t5_xxl_fp16.

For more details, visit: 

🌟The following FLUX-IP-Adapter Workflow is specifically designed for the  model.🌟

About FLUX-IP-Adapter

FLUX-IP-Adapter allows users to generate images by adapting pre-trained models to specific image styles or concepts. The FLUX-IP-Adapter model is trained on both 512x512 and 1024x1024 resolutions, making it versatile for various image generation tasks.

How to Use ComfyUI FLUX-IP-Adapter Workflow

To use the FLUX-IP-Adapter in ComfyUI, follow these steps:

1. Load the FLUX-IP-Adapter Model

• Use the "Flux Load IPAdapter" node in the ComfyUI workflow.
• Select the appropriate FLUX-IP-Adapter model file (e.g., "flux-ip-adapter.safetensors").
• Select the appropriate clip vision (e.g., "clip_vision_l.safetensors").
• Choose the desired device for running the model (e.g., CPU or GPU).

2. Apply the FLUX-IP-Adapter to the Base Model

• Connect the output of the "Flux Load IPAdapter" node to the "Apply Flux IPAdapter" node.
• Load the base model using the "UNETLoader" node and connect its output to the "Apply Flux IPAdapter" node.
• Set the desired mix strength (e.g., 0.92) in the "Apply Flux IPAdapter" node to control the influence of the IP-Adapter on the base model.

3. Provide Input Image and Text Conditioning

• Load an input image using the "LoadImage" node and connect its output to the "ImageScale" node.
• Scale the input image to the desired resolution using the "ImageScale" node.
• Connect the output of the "ImageScale" node to the "Apply Flux IPAdapter" node.
• Use the "CLIPTextEncodeFlux" node to provide text conditioning for the image generation process. Enter the desired text prompt and connect its output to the "XlabsSampler" node.

4. Configure Sampling Settings and Generate

• In the "XlabsSampler" node, set the following parameters:
  • Sampling Steps: The number of steps for the sampling process (e.g., 50).
  • Sampler: The sampling method to use (e.g., "fixed").
  • CFG Scale: The Classifier-Free Guidance scale to control the influence of the text conditioning (e.g., 3.5).
  • Seed: The random seed for reproducibility (set to "-1" for random seed).

License: FLUX-IP-Adapter model weights fall under the  Non-Commercial License

License

View license files:

The FLUX.1 [dev] Model is licensed by Black Forest Labs. Inc. under the FLUX.1 [dev] Non-Commercial License. Copyright Black Forest Labs. Inc.

IN NO EVENT SHALL BLACK FOREST LABS, INC. BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH USE OF THIS MODEL.