ComfyUI Node: Ultralytics Inference

Ultralytics Inference:

The UltralyticsInference node is designed to perform object detection and image analysis using pre-trained models from the Ultralytics YOLO series. This node leverages the power of YOLO (You Only Look Once) models to quickly and accurately identify objects within an image, making it an essential tool for AI artists who need to incorporate advanced image recognition capabilities into their projects. By using this node, you can detect various objects in an image, obtain their bounding boxes, and extract other relevant information such as masks, probabilities, and keypoints. The node is highly configurable, allowing you to adjust parameters like confidence threshold, intersection over union (IoU) threshold, image dimensions, and more, to fine-tune the detection process according to your specific needs.

Ultralytics Inference Input Parameters:

model

This parameter expects an Ultralytics model that has been loaded using the UltralyticsModelLoader or CustomUltralyticsModelLoader nodes. The model is used to perform the inference on the provided image.

image

This parameter takes an image input on which the object detection will be performed. The image should be in a compatible format that the model can process.

conf

This is the confidence threshold for object detection. It determines the minimum confidence level required for a detection to be considered valid. The value ranges from 0 to 1, with a default of 0.25. Lowering this value may result in more detections, including less confident ones, while increasing it will filter out less certain detections.

iou

The intersection over union (IoU) threshold is used for non-maximum suppression, which helps in eliminating redundant overlapping boxes. The value ranges from 0 to 1, with a default of 0.7. A higher value will result in fewer boxes, while a lower value will allow more overlapping boxes.

height

This parameter sets the height of the image to be used for inference. The value ranges from 64 to 1280 pixels, with a default of 640 pixels. Adjusting this value can impact the accuracy and speed of the detection.

width

This parameter sets the width of the image to be used for inference. The value ranges from 64 to 1280 pixels, with a default of 640 pixels. Similar to the height parameter, changing this value can affect the detection performance.

device

This parameter specifies the device to be used for inference, with options including cuda:0 for GPU and cpu for CPU. Using a GPU can significantly speed up the inference process.

half

This boolean parameter determines whether to use half-precision floating-point numbers during inference. The default value is False. Enabling this option can reduce memory usage and potentially increase inference speed on compatible hardware.

augment

This boolean parameter indicates whether to use data augmentation during inference. The default value is False. Enabling augmentation can improve detection robustness but may increase inference time.

agnostic_nms

This boolean parameter specifies whether to use class-agnostic non-maximum suppression. The default value is False. When enabled, this option will apply non-maximum suppression across all classes, which can be useful in certain scenarios.

classes

This parameter allows you to specify a comma-separated list of class names to filter the detections. The default value is "None", meaning all classes will be considered. Providing specific class names will limit the detections to those classes only.

Ultralytics Inference Output Parameters:

ULTRALYTICS_RESULTS

This output contains the overall results of the inference, including detected objects and their associated data.

IMAGE

This output provides the original image with the detected objects overlaid, allowing you to visualize the results directly.

BOXES

This output contains the bounding boxes for the detected objects, represented as coordinates in the image.

MASKS

This output includes the segmentation masks for the detected objects, if available.

PROBABILITIES

This output provides the confidence scores for each detected object, indicating the likelihood of each detection being correct.

KEYPOINTS

This output contains keypoints for the detected objects, which can be useful for tasks requiring detailed object analysis.

OBB

This output includes oriented bounding boxes for the detected objects, providing additional spatial information.

LABELS

This output provides the class labels for the detected objects, indicating the type of each detected object.

Ultralytics Inference Usage Tips:

• Adjust the conf parameter to balance between detection sensitivity and precision. Lower values may detect more objects but include false positives, while higher values will be more selective.
• Use the device parameter to leverage GPU acceleration for faster inference times, especially when processing large images or batches.
• Experiment with the height and width parameters to find the optimal image size for your specific use case, balancing between detection accuracy and processing speed.
• Enable augment if you need more robust detections in varied conditions, but be aware that it may increase inference time.

Ultralytics Inference Common Errors and Solutions:

"Model not loaded"

• Explanation: This error occurs when the model parameter is not provided or the model failed to load correctly.
• Solution: Ensure that you have loaded a valid Ultralytics model using the UltralyticsModelLoader or CustomUltralyticsModelLoader nodes before running the inference.

"Invalid image format"

• Explanation: This error indicates that the provided image is in an unsupported format or corrupted.
• Solution: Verify that the image is in a compatible format and not corrupted. Convert the image to a supported format if necessary.

"CUDA device not available"

• Explanation: This error occurs when the specified CUDA device is not available or not properly configured.
• Solution: Check your system's GPU configuration and ensure that the CUDA drivers are installed correctly. You can also switch to cpu if GPU is not available.

"Invalid parameter value"

• Explanation: This error is raised when one or more input parameters have values outside the acceptable range.
• Solution: Review the input parameter values and ensure they fall within the specified ranges. Adjust any values that are out of bounds.