ComfyUI Node: Intrinsic Lora Sampling

Intrinsic Lora Sampling:

The Intrinsic_lora_sampling node is designed to facilitate the use of intrinsic LoRAs (Low-Rank Adaptations) in your AI art projects. This node allows you to apply specific intrinsic LoRAs to your models, enabling the generation of various intrinsic images such as depth maps, surface normals, albedo, and shading. By leveraging these LoRAs, you can enhance the visual quality and detail of your generated images. The node integrates seamlessly with your existing models and provides a streamlined process for sampling and applying these intrinsic LoRAs, making it a powerful tool for AI artists looking to add depth and realism to their creations.

Intrinsic Lora Sampling Input Parameters:

model

This parameter specifies the model to which the intrinsic LoRA will be applied. It is essential for defining the base model that will be enhanced with the LoRA.

lora_name

This parameter allows you to select the specific intrinsic LoRA to be used. The available options are listed from the intrinsic LoRAs folder, and you can choose the one that best fits your task.

task

This parameter defines the type of intrinsic image you want to generate. The available options are depth map, surface normals, albedo, and shading. The default value is depth map.

text

This parameter allows you to input a string of text that can be used for conditioning the model. It supports multiline input and has a default empty string.

clip

This parameter specifies the CLIP model to be used for text encoding. It is necessary for processing the text input and generating the corresponding conditioning.

vae

This parameter specifies the VAE (Variational Autoencoder) model to be used for encoding and decoding images. It is crucial for handling the image data during the sampling process.

per_batch

This parameter defines the number of samples to process per batch. It has a default value of 16, with a minimum of 1 and a maximum of 4096. Adjusting this value can impact the performance and speed of the node.

image (optional)

This optional parameter allows you to input an image that can be used as a starting point for the sampling process. If not provided, the node will generate samples from scratch.

optional_latent (optional)

This optional parameter allows you to input a latent representation that can be used instead of generating new samples. It provides flexibility in the sampling process by allowing you to reuse existing latent data.

Intrinsic Lora Sampling Output Parameters:

IMAGE

This output parameter provides the generated intrinsic image based on the selected task. The image is processed and enhanced using the specified intrinsic LoRA, resulting in a detailed and visually appealing output.

LATENT

This output parameter provides the latent representation of the generated samples. It can be used for further processing or as input for other nodes in your workflow.

Intrinsic Lora Sampling Usage Tips:

• To achieve the best results, carefully select the intrinsic LoRA that matches your desired task, such as depth map or surface normals.
• Adjust the per_batch parameter based on your system's capabilities to optimize performance and speed.
• Utilize the text parameter to condition the model with specific prompts, enhancing the relevance and quality of the generated images.
• Experiment with providing an image or optional_latent input to see how it influences the final output and to leverage existing data.

Intrinsic Lora Sampling Common Errors and Solutions:

Error: "LoRA file not found"

• Explanation: This error occurs when the specified LoRA file cannot be located in the intrinsic LoRAs folder.
• Solution: Ensure that the LoRA file exists in the correct folder and that the lora_name parameter is correctly specified.

Error: "Invalid task type"

• Explanation: This error occurs when an unsupported task type is selected.
• Solution: Verify that the task parameter is set to one of the supported options: depth map, surface normals, albedo, or shading.

Error: "Model or VAE not specified"

• Explanation: This error occurs when the required model or vae parameters are not provided.
• Solution: Ensure that both the model and vae parameters are specified and correctly configured.

Error: "Batch size out of range"

• Explanation: This error occurs when the per_batch parameter is set to a value outside the allowed range.
• Solution: Adjust the per_batch parameter to a value between 1 and 4096.