ComfyUI Node: MarigoldDepthEstimation_v2

MarigoldDepthEstimation_v2:

MarigoldDepthEstimation_v2 is an advanced node designed for diffusion-based monocular depth estimation, which is a technique used to infer depth information from a single image. This node leverages the Marigold model to generate depth maps, which are essential for various applications such as 3D reconstruction, augmented reality, and image editing. The node is particularly beneficial for AI artists as it provides a way to create depth maps with high accuracy and consistency. It includes features like optical flow for video consistency, making it suitable for processing video frames to ensure smooth transitions between them. The node is experimental and offers various parameters to fine-tune the depth estimation process, allowing you to balance between processing time and accuracy.

MarigoldDepthEstimation_v2 Input Parameters:

denoise_steps

This parameter determines the number of steps per depth map. Increasing the number of denoise steps can enhance the accuracy of the depth map but will also increase the processing time. There is no strict minimum or maximum value, but typical values range from 1 to 10, with a default of around 4 steps for the LCM model.

n_repeat

This parameter specifies the number of iterations to be ensembled into a single depth map. Higher values can improve the quality of the depth map by reducing noise, but will also require more computational resources. Typical values range from 1 to 10, with a default of 1.

n_repeat_batch_size

This parameter defines how many of the n_repeats are processed as a batch. If you have sufficient VRAM, setting this to match n_repeats can speed up processing. The default value is usually 1, but it can be increased based on available VRAM.

model

This parameter allows you to choose between the Marigold model and its LCM version, marigold-lcm-v1-0. The LCM model is optimized for fewer steps and should be used with the LCMScheduler. The default model is Marigold.

scheduler

Different schedulers can produce slightly different results. This parameter allows you to select the scheduler that best fits your needs. There is no default value as it depends on the specific requirements of your task.

invert

By default, Marigold produces a depth map where black represents the front. This parameter allows you to invert the depth map, which is useful for applications like controlnets where the opposite representation is needed. The default value is false.

regularizer_strength

This parameter controls the strength of the regularizer in the ensembling process. It is generally recommended not to modify this setting unless you have specific requirements. The default value is typically set by the model.

reduction_method

This parameter specifies the method used for reducing the depth maps during the ensembling process. It is generally recommended not to modify this setting. The default value is typically set by the model.

max_iter

This parameter sets the maximum number of iterations for the ensembling process. It is generally recommended not to modify this setting. The default value is typically set by the model.

tol

This parameter defines the tolerance level for the ensembling process. It is generally recommended not to modify this setting. The default value is typically set by the model.

use_fp16

This parameter allows you to choose between fp16 and fp32 precision. Using fp16 can significantly reduce VRAM usage but may lead to a loss of quality in some cases. The default value is false, meaning fp32 is used.

MarigoldDepthEstimation_v2 Output Parameters:

ensembled_image

The output of this node is an ensembled depth map image. This image represents the depth information inferred from the input image, where different shades indicate varying distances from the camera. The depth map can be used for various applications such as 3D reconstruction, augmented reality, and image editing. The output is crucial for creating realistic depth effects and enhancing the visual quality of your projects.

MarigoldDepthEstimation_v2 Usage Tips:

• For higher accuracy, increase the denoise_steps parameter, but be aware that this will also increase processing time.
• If you have sufficient VRAM, set n_repeat_batch_size to match n_repeat for faster processing.
• Use the invert parameter if you need a depth map where white represents the front, which is useful for controlnets.
• Experiment with different scheduler options to find the one that best fits your specific needs.
• Use use_fp16 to reduce VRAM usage, but check the output quality to ensure it meets your standards.

MarigoldDepthEstimation_v2 Common Errors and Solutions:

"CUDA out of memory"

• Explanation: This error occurs when the GPU runs out of memory while processing the depth map.
• Solution: Reduce the n_repeat_batch_size or denoise_steps parameters to lower the memory usage. Alternatively, use the use_fp16 parameter to reduce VRAM consumption.

"Invalid scheduler selected"

• Explanation: This error occurs when an unsupported scheduler is chosen.
• Solution: Ensure that the scheduler you select is compatible with the model you are using. Refer to the documentation for a list of supported schedulers.

"Depth map values out of range"

• Explanation: This error occurs when the depth map values are not within the expected range.
• Solution: Check the invert parameter and ensure it is set correctly. Verify that the input image is suitable for depth estimation.

"Model not found"

• Explanation: This error occurs when the specified model is not available.
• Solution: Ensure that the model name is correct and that the model files are properly installed in the expected directory.