Chat LLM

Description

Chat LLM component allows prompting LLM chat models. It is available once at least one AI integration is configured in the Admin Panel.

Parameters and configuration

Name	Description
LLM Integration	LLM integration to use
Chat model name	The chat model to use. The available models depend on the LLM Integration selected
Prompt	Description of the concrete task to be performed.
Output schema	Optional; a JSON Schema describing the desired response. When provided, the returned value becomes a typed record instead of plain text — see Structured output below.
Temperature	Temperature is a hyperparameter that controls the randomness of text generation. Lower values (e.g., 0.2) make the model’s output more focused and deterministic by favoring the most likely tokens, while higher values (e.g., 0.8 or above) produce more diverse and creative responses by flattening the probability distribution.
Output variable name	Variable name under which node result will be available in a subsequent nodes.

Returned value

By default the component returns the chat model's response as plain text (String).

When an Output schema is provided, the returned value becomes a typed record with raw and structured fields instead — see Structured output below.

Structured output

Provide a JSON Schema in the Output schema parameter to get a typed response instead of plain text. Nussknacker uses the schema in two ways:

• At runtime, the schema is sent to the model as a response-format constraint, steering it to answer in exactly that shape.
• At design time, the schema is converted into a Nussknacker type, so the rest of the scenario knows the fields and their types — with autocomplete and validation — before the scenario runs.

With a schema, the output variable becomes a record with two fields:

Field	Type	Description
raw	String	The model's response as text, exactly as it came back — parseable or not.
structured	the schema's type, or null	The response parsed and typed against the schema, or null when the response is not valid JSON or does not match the schema.

For example, this schema:

{
  "type": "object",
  "properties": {
    "brand":     { "type": "string" },
    "price":     { "type": "number", "minimum": 0 },
    "sentiment": { "type": "string", "enum": ["positive", "neutral", "negative"] }
  },
  "required": ["brand", "price", "sentiment"]
}

types the output (for an output variable named #output) as:

#output.raw                  : String
#output.structured           : Record{brand: String, price: BigDecimal, sentiment: String(negative) | String(neutral) | String(positive)}
#output.structured.price     : BigDecimal
#output.structured.sentiment : String(negative) | String(neutral) | String(positive)

Handling invalid responses

LLMs are probabilistic, so a model may occasionally return something that is not valid JSON or does not match the schema. When that happens, structured is null — a value you can branch on, rather than an error that stops the scenario:

• raw always holds the original text, so the response is never lost.
• Filter on #output.structured != null to separate successful extractions from failures and handle them separately (for example, route failures to a dead-letter sink), while the rest of the flow keeps running.

Additional considerations

• Without an Output schema, the response is plain text; if you expect JSON you can parse it manually using #CONV.toJson() or #CONV.toJsonOrNull(). With a schema, use the typed structured field instead of parsing by hand.
• Chat models may not follow the provided output structure - models can even not return a valid json in some cases.
• You can use string template during prompt creation to use data available in the scenario (e.g., documents retrieved from the vector store)
• Currently only selected models are available. Contact us if you need to use additional ones.