ComfyUI Node: MarigoldDepthEstimation

MarigoldDepthEstimation:

MarigoldDepthEstimation is a powerful node designed for diffusion-based monocular depth estimation, which allows you to generate depth maps from single images. This node leverages advanced machine learning techniques to produce highly accurate depth maps, which can be particularly useful for various applications in AI art, such as creating 3D effects, enhancing image realism, or generating depth-aware artistic transformations. The node is capable of handling different models and schedulers, providing flexibility in terms of accuracy and processing time. Additionally, it includes options for ensembling multiple iterations to refine the depth map and supports both fp16 and fp32 precision modes to balance VRAM usage and output quality.

MarigoldDepthEstimation Input Parameters:

denoise_steps

This parameter defines the number of steps per depth map. Increasing the number of denoise steps can enhance the accuracy of the depth map but will also increase the processing time. There is no strict minimum or maximum value, but typical values range from 1 to 10, with a default value often set around 4 for a good balance between speed and accuracy.

n_repeat

This parameter specifies the number of iterations to be ensembled into a single depth map. Higher values can improve the quality of the depth map by averaging out noise and inconsistencies, but will also require more computational resources. Common values range from 1 to 10, with a default value of 3.

n_repeat_batch_size

This parameter determines how many of the n_repeats are processed as a batch. If you have sufficient VRAM, setting this value equal to n_repeat can speed up processing. The default value is typically 1, but it can be adjusted based on available VRAM.

model

This parameter allows you to choose between the Marigold model and its LCM version, marigold-lcm-v1-0. The LCM model is optimized for fewer steps and works best with the LCMScheduler. The default model is usually Marigold.

scheduler

Different schedulers can produce slightly different results. This parameter lets you select the scheduler that best fits your needs. Common options include the default scheduler and the LCMScheduler for the LCM model.

invert

By default, Marigold produces a depth map where black represents the front. This parameter allows you to invert the depth map, which is useful for applications like controlnets that require the opposite representation. The default value is false.

regularizer_strength

This parameter controls the strength of the regularizer used in the ensembling process. It generally does not need to be adjusted from its default value, which is optimized for most use cases.

reduction_method

This parameter specifies the method used to reduce multiple depth maps into a single ensembled depth map. The default method is usually sufficient for most applications.

max_iter

This parameter sets the maximum number of iterations for the ensembling process. The default value is typically set to ensure a good balance between quality and processing time.

tol

This parameter defines the tolerance level for the ensembling process. It generally does not need to be adjusted from its default value.

use_fp16

This parameter allows you to choose between fp16 and fp32 precision modes. Using fp16 can significantly reduce VRAM usage but may lead to a slight loss in quality. The default value is false, meaning fp32 is used.

MarigoldDepthEstimation Output Parameters:

ensembled_image

The ensembled_image is the final output of the node, representing the depth map generated from the input image. This depth map can be used for various artistic and technical applications, such as creating 3D effects, enhancing image realism, or generating depth-aware transformations. The depth map is typically represented as a grayscale image where different shades indicate different depths.

MarigoldDepthEstimation Usage Tips:

• For higher accuracy, increase the denoise_steps parameter, but be aware that this will also increase processing time.
• If you have sufficient VRAM, set n_repeat_batch_size equal to n_repeat to speed up processing.
• Use the invert parameter if your application requires a depth map where white represents the front.
• Experiment with different schedulers to find the one that produces the best results for your specific use case.
• If you encounter VRAM limitations, consider using the use_fp16 parameter to reduce memory usage.

MarigoldDepthEstimation Common Errors and Solutions:

"CUDA out of memory"

• Explanation: This error occurs when the GPU runs out of memory while processing the depth map.
• Solution: Reduce the n_repeat_batch_size or use the use_fp16 parameter to lower VRAM usage.

"Invalid model selected"

• Explanation: This error occurs when an unsupported model is specified in the model parameter.
• Solution: Ensure that the model parameter is set to either "Marigold" or "marigold-lcm-v1-0".

"Scheduler not found"

• Explanation: This error occurs when an unsupported scheduler is specified.
• Solution: Verify that the scheduler parameter is set to a valid option, such as the default scheduler or LCMScheduler.

"Depth map inversion failed"

• Explanation: This error occurs if there is an issue with inverting the depth map.
• Solution: Double-check the invert parameter and ensure it is set correctly. If the problem persists, try processing the depth map without inversion and manually invert it later if needed.