ComfyUI Node: LoRA Merge 🍌

LoRA Merge 🍌| LoRA Merge 🍌:

LoraMerger| LoRA Merge 🍌 is a specialized node designed to merge two Low-Rank Adaptation (LoRA) models, which are commonly used in AI art generation to fine-tune large pre-trained models. This node allows you to combine the strengths of two different LoRA models into a single, more powerful model. By leveraging different merging modes such as addition, concatenation, and Singular Value Decomposition (SVD), LoraMerger| LoRA Merge 🍌 provides flexibility in how the models are integrated. This can be particularly useful for artists looking to blend different styles or enhance specific features in their AI-generated artwork. The node also supports various configurations for rank, threshold, device, and data type, making it adaptable to different computational environments and precision requirements.

LoRA Merge 🍌| LoRA Merge 🍌 Input Parameters:

lora_1

This is the first LoRA model that you want to merge. It is a required input and serves as the primary model in the merging process.

lora_2

This is the second LoRA model that you want to merge with the first one. It is an optional input, and if not provided, the node will only use lora_1 for the merging process.

mode

This parameter determines the merging strategy to be used. The available options are "add", "concat", and "svd". "Add" will sum the weights of the two models, "concat" will concatenate them, and "svd" will use Singular Value Decomposition for a more sophisticated merging. The default value is "add".

rank

This integer parameter specifies the rank for the SVD merging mode. It controls the dimensionality of the merged model. The default value is 16, with a minimum of 1 and a maximum of 320. This parameter is only relevant when the merging mode is set to "svd".

threshold

This floating-point parameter sets the threshold for the SVD merging mode. It helps in determining the significance of singular values during the decomposition. The default value is 1.0, with a range from 0 to 1, and a step size of 0.01. This parameter is only relevant when the merging mode is set to "svd".

device

This parameter specifies the computational device to be used for the merging process. The available options are "cuda" for GPU and "cpu" for CPU. The default value is "cuda".

dtype

This parameter sets the data type for the merging process. The available options are "float32", "float16", and "bfloat16". The default value is "float32".

LoRA Merge 🍌| LoRA Merge 🍌 Output Parameters:

LoRA

The output is a single merged LoRA model. This model combines the features and strengths of the input LoRA models based on the specified merging mode and parameters. It can be used for further fine-tuning or directly in AI art generation tasks.

LoRA Merge 🍌| LoRA Merge 🍌 Usage Tips:

• For a straightforward merging of two models, use the "add" mode with default parameters.
• If you want to combine models with different styles, consider using the "concat" mode.
• For more advanced merging, use the "svd" mode and experiment with different rank and threshold values to achieve the desired balance between the two models.
• Ensure that your computational device (CPU or GPU) is appropriately selected based on your hardware capabilities to optimize performance.

LoRA Merge 🍌| LoRA Merge 🍌 Common Errors and Solutions:

"KeyError: 'lora_down' not found in lora_1 or lora_2"

• Explanation: This error occurs when the specified key for the LoRA model is not found in one or both of the input models.
• Solution: Ensure that both lora_1 and lora_2 are correctly loaded and contain the necessary keys for merging.

"ValueError: Invalid merging mode"

• Explanation: This error occurs when an unsupported merging mode is specified.
• Solution: Check the mode parameter and ensure it is set to one of the supported values: "add", "concat", or "svd".

"RuntimeError: CUDA out of memory"

• Explanation: This error occurs when the GPU does not have enough memory to perform the merging operation.
• Solution: Reduce the rank or threshold values, or switch to using the CPU by setting the device parameter to "cpu".

"TypeError: Unsupported dtype"

• Explanation: This error occurs when an unsupported data type is specified.
• Solution: Ensure that the dtype parameter is set to one of the supported values: "float32", "float16", or "bfloat16".