ComfyUI Node: LLM Chat

LLM Chat:

The AV_LLMChat node is designed to facilitate seamless interaction with various Large Language Models (LLMs) by enabling chat-based communication. This node allows you to send a series of messages to an LLM and receive a coherent response, making it ideal for creating interactive AI-driven dialogues. Whether you are developing a conversational AI, generating creative content, or seeking assistance from an AI model, the AV_LLMChat node provides a streamlined interface to leverage the power of LLMs. By configuring the node with appropriate parameters, you can customize the behavior of the chat model to suit your specific needs, ensuring a tailored and effective AI interaction.

LLM Chat Input Parameters:

messages

The messages parameter is a list of messages that you want to send to the LLM. Each message should be an instance of LLM_MESSAGE, which includes the role (system, user, or assistant) and the text content. This parameter is crucial as it forms the basis of the conversation with the LLM, guiding the model on how to respond based on the provided context.

api

The api parameter specifies the API instance that will be used to communicate with the LLM. This should be an instance of LLM_API, such as OpenAI or Claude API. The API instance determines which LLM will process the messages and generate the response, impacting the style and quality of the interaction.

config

The config parameter is an instance of LLM_CONFIG that includes various settings for the LLM, such as the model to be used, the maximum number of tokens in the response, and the temperature (which controls the randomness of the output). Proper configuration of this parameter allows you to fine-tune the behavior of the LLM to meet your specific requirements.

seed

The seed parameter is an integer value used to initialize the random number generator for the LLM. This can help in generating reproducible results. The default value is 0, and it can range from 0 to 0x1FFFFFFFFFFFFF. Setting this parameter can be useful if you want to ensure consistency in the responses generated by the LLM.

LLM Chat Output Parameters:

response

The response parameter is a string that contains the text generated by the LLM in response to the provided messages. This output is the result of the chat interaction and can be used directly in your application to display the AI's response or further process it as needed. The quality and relevance of the response depend on the input messages and the configuration settings.

LLM Chat Usage Tips:

• Ensure that the messages parameter includes a clear and coherent sequence of messages to guide the LLM effectively.
• Use the config parameter to fine-tune the model's behavior, such as adjusting the temperature to control the creativity of the responses.
• Set the seed parameter if you need reproducible results, especially during testing and development phases.
• Choose the appropriate api instance based on the desired LLM, as different models may have varying strengths and response styles.

LLM Chat Common Errors and Solutions:

Must provide a Claude v3 model, got {config.model}

• Explanation: This error occurs when the specified model in the config parameter is not a Claude v3 model.
• Solution: Ensure that the model in the config parameter is set to a valid Claude v3 model.

OpenAI API key is required

• Explanation: This error indicates that the OpenAI API key is missing or not provided.
• Solution: Provide a valid OpenAI API key in the api parameter or set it as an environment variable.

System prompt does not support image

• Explanation: This error occurs when an image is included in a system message, which is not supported.
• Solution: Remove the image from the system message or change the role of the message to user or assistant.

Only one system prompt is allowed

• Explanation: This error indicates that multiple system messages are included in the messages parameter.
• Solution: Ensure that there is only one system message in the messages list.

Error: {error_message}

• Explanation: This is a generic error message indicating an issue with the API request or response.
• Solution: Check the details of the error message and ensure that all parameters are correctly set and the API endpoint is reachable.