AI Models Services Providers
Overview
This group of integrations allows the use of AI models in Nussknacker scenarios. Chat models will be available via the LLM Chat component and embedding models will be available via the Vector Store component.
Currently, the following AI models providers are supported: Amazon Bedrock, Anthropic, Google AI Gemini, MistralAI, Nvidia, OpenAI, and any server with the OpenAI API (OpenAI-compatible).
Anthropic provides chat models only. An Anthropic integration can be used in the LLM Chat and AI Agent components, but not for embedding models in the Vector Store component. When the Max output tokens parameter of a component is left empty, Anthropic models use a limit of 8192 tokens.
To add an AI model integration, open Integrations from the Designer's side menu and click Add new on the provider's tile.
For typed choices, scores and yes/no probabilities, configure a Decision AI integration and use the Decision Model component. It currently supports Jev through System One.
Integration parameters
| Name | Description |
|---|---|
| Integration name | The integration name will be used to refer to this configuration in LLM related enrichers, e.g. LLM Chat. |
| API key | API key to access the AI models provider, e.g. an Anthropic API key for the Anthropic integration. API key validity can be checked using the Check Connection button. |
The same two parameters are used for every provider listed above, including Anthropic. The OpenAI-compatible integration takes the ones described below.
OpenAI-compatible servers
The OpenAI-compatible integration connects any server that offers the OpenAI API: a model you run yourself with Ollama or vLLM, or a service such as OpenRouter, Groq or DeepSeek. Such a server has no fixed list of models, so the integration names the ones to use.
| Name | Description |
|---|---|
| URL | Base URL of the server's OpenAI API, e.g. http://localhost:11434/v1 for Ollama or http://localhost:8000/v1 for vLLM. |
| API key | Optional; only for a server that requires one. |
| Chat models | Models offered in the LLM Chat and AI Agent nodes. |
| Embedding models | Models offered for embeddings in the Vector Store node. |
Click Check Connection to list the models available on the server, then choose them in the model fields. You can
also type in a model name the server doesn't list. At least one chat or embedding model is required.
The server's models differ in what they can do. A model may not follow the output schema of the LLM Chat node, and
the AI Agent needs a model that can call tools. The output token limit is sent as max_tokens, which these servers
understand.