ComfyUI Node: Core ML Sampler (Advanced)

Core ML Sampler (Advanced):

The CoreMLSamplerAdvanced node is a sophisticated tool designed to enhance your AI art generation process by leveraging advanced sampling techniques within the Core ML framework. This node integrates the capabilities of Core ML with the advanced features of the KSamplerAdvanced class, allowing for more refined and controlled sampling of latent images. It is particularly useful for generating high-quality images with precise control over various parameters such as noise, steps, and conditioning. By using this node, you can achieve better results in your AI art projects, especially when working with complex models like LCM (Latent Control Models) and SDXL (Stable Diffusion XL).

Core ML Sampler (Advanced) Input Parameters:

coreml_model

This parameter specifies the Core ML model to be used for sampling. It is essential for defining the model architecture and weights that will guide the image generation process. The model must be compatible with Core ML and can include advanced features like LCM and SDXL.

add_noise

This boolean parameter determines whether to add noise to the latent image during the sampling process. Adding noise can help in generating more diverse and creative outputs. The default value is typically True.

noise_seed

This parameter sets the seed for the noise generation process. By controlling the seed, you can ensure reproducibility of the generated images. It accepts integer values, and using the same seed will produce the same noise pattern.

steps

This parameter defines the number of steps to be taken during the sampling process. More steps generally lead to higher quality images but will increase the computation time. Typical values range from 10 to 1000, depending on the desired quality and available computational resources.

cfg

The cfg parameter stands for Classifier-Free Guidance, which controls the strength of the guidance applied during sampling. Higher values result in images that more closely follow the provided conditioning, while lower values allow for more creative freedom. The value is usually a float between 0.0 and 20.0.

sampler_name

This parameter specifies the name of the sampler to be used. Different samplers can produce different styles and qualities of images. Common options include ddim, plms, and heun.

scheduler

The scheduler parameter defines the scheduling strategy for the sampling steps. It can impact the convergence and quality of the generated images. Options may include linear, cosine, and exponential.

positive

This parameter provides the positive conditioning for the model, guiding it towards desired features in the generated image. It is typically a list of tuples, where each tuple contains a conditioning vector and associated metadata.

start_at_step

This parameter specifies the step at which to start the sampling process. It allows for partial sampling, which can be useful for refining previously generated images. The value is an integer, usually starting from 0.

end_at_step

This parameter defines the step at which to end the sampling process. It allows for early stopping, which can save computation time if the desired quality is achieved before the maximum number of steps. The value is an integer, typically up to the maximum number of steps.

return_with_leftover_noise

This boolean parameter determines whether to return the generated image along with any leftover noise. This can be useful for further processing or analysis. The default value is False.

negative

This optional parameter provides negative conditioning for the model, guiding it away from undesired features in the generated image. It is typically a list of tuples, similar to the positive parameter. For LCM models, this parameter is optional.

latent_image

This parameter specifies the initial latent image to be used for sampling. If not provided, a default latent image will be generated. It can be useful for refining or continuing the generation of a previously created image.

denoise

The denoise parameter controls the amount of denoising applied during the sampling process. Higher values result in smoother images, while lower values retain more noise and texture. The value is usually a float between 0.0 and 1.0.

Core ML Sampler (Advanced) Output Parameters:

sampled_image

The sampled_image parameter is the primary output of the node, representing the generated image after the sampling process. This image is influenced by all the input parameters and reflects the model's interpretation of the provided conditioning and noise.

leftover_noise

If the return_with_leftover_noise parameter is set to True, this output will contain any leftover noise from the sampling process. This can be useful for further analysis or iterative refinement of the generated image.

Core ML Sampler (Advanced) Usage Tips:

• Experiment with different cfg values to find the right balance between adherence to conditioning and creative freedom.
• Use the noise_seed parameter to ensure reproducibility of your results, especially when fine-tuning the model.
• Adjust the steps parameter based on your computational resources and desired image quality; more steps generally yield better results.
• Leverage the start_at_step and end_at_step parameters for partial sampling, which can save time and resources when refining images.

Core ML Sampler (Advanced) Common Errors and Solutions:

"Negative conditioning is optional only for LCM models."

• Explanation: This error occurs when the negative parameter is not provided for non-LCM models.
• Solution: Ensure that you provide a valid negative conditioning parameter unless you are using an LCM model.

"Model not compatible with Core ML."

• Explanation: This error indicates that the provided model is not compatible with Core ML.
• Solution: Verify that the model is correctly converted to Core ML format and is compatible with the Core ML framework.

"Invalid sampler name."

• Explanation: This error occurs when an unsupported sampler name is provided.
• Solution: Check the available sampler names and ensure you are using a valid option like ddim, plms, or heun.

"Steps parameter out of range."

• Explanation: This error indicates that the steps parameter is set outside the acceptable range.
• Solution: Adjust the steps parameter to a value within the typical range of 10 to 1000.

"Denoise value out of range."

• Explanation: This error occurs when the denoise parameter is set outside the range of 0.0 to 1.0.
• Solution: Ensure that the denoise parameter is set to a float value between 0.0 and 1.0.