Skip to main content
Version: Current

Topics

Version: Nussknacker Cloud Mode: Streaming

Description​

The Topics page lets you manage the Kafka topics your scenarios read from and write to, together with their message schemas, without leaving the Designer. Open it with the Topics item of the Designer side menu (path /topics). The page is available when the Topics extension is enabled on your installation (it is in Nussknacker Cloud and in the enterprise installation bundle); see Configuration below.

The topics listed here live in the Kafka cluster the Topics extension is configured with. They are the topics offered by the Topic source and sink (see Topic (Source) and Topic (Sink)), so you can use them to pass events between scenarios. Kafka and the schema registry are the source of truth: the page only reads and changes what is there.

Kafka clusters​

Besides the Kafka dedicated to Nussknacker, the page manages the topics of every other Kafka the scenarios use, such as one added under Integrations. With more than one, a Kafka field above the topic list picks the cluster; the dedicated one is shown first. The choice is part of the page address, so a link to a topic opens it on its own cluster. The topics of a Kafka added under Integrations are named as in Kafka, without the instance's namespace, and Used by lists the nodes of that cluster's own source and sink. A topic created there gets the partitions, replication factor and maximum message size of that Kafka's broker, and does not count against the topic limit of this instance.

Nussknacker reaches another Kafka with the credentials set for it, which may allow less than your own permissions do. The page shows what that Kafka reports about them: Add topic, Clone, Edit, Delete and the Messages tab are greyed out where the credentials do not allow it, and hovering over them says so. The list holds only the topics those credentials can see. When Kafka or its schema registry refuses an operation anyway, the page shows its reason instead of a generic error.

Topics list​

The page lists the topics on the left and shows the selected one on the right, like the Key-Value Store page. The list shows every topic of this instance, sorted by name, with Schema or No schema and its retention (<n> h, or Unlimited retention). Search it by name, click a topic to open its details, or use Add topic above the list to create a new one.

Adding a topic​

Add topic opens the Add new topic dialog. The name field is prefilled with a random name, which you can change.

FieldDescription
Topic nameName of the topic. Allowed characters: letters, digits, ., _ and -; at most 200 characters. It must not be the name of an existing topic.
Message schemaNo schema, JSON Schema or Avro. With No schema the topic accepts any message, and Nussknacker cannot validate field names and data types of SpEL expressions using it; for JSON messages you can still use dynamic field access. Read this section to understand the role of a topic schema.
SchemaShown when a schema type is selected; required then. The JSON Schema or Avro schema of the message value, written as JSON.
CompatibilityShown when a schema type is selected. What the schema registry checks the next version of the schema against, see Schema compatibility. Backward by default, as in the schema registry.
Retention time (hours)For how many hours a message stays available for consumers after being published; a positive integer. Left empty, a new topic gets the default retention set by the administrator, and an edited topic keeps its retention, also an unlimited one. A retention shorter than the Kafka cluster's segment time (7 days by default) also sets the topic's segment time to it, so old messages are deleted on time: a message stays at most about twice the retention.

Click Add topic to create it. The number of partitions, the replication factor and the maximum message size are not set per topic: they come from the configuration.

Topic details​

The details of a topic show its name, the actions and the description on top, and the rest in tabs - Schema, Messages and Used by, like the details of a Fluss table:

  • Description - what the topic carries, kept in its schema: the doc of an Avro schema or the description of a JSON schema. Users allowed to edit topics change it with the pencil next to it; saving registers a new schema version that differs only in the description. A topic without a schema has no description.
  • Schema - the fields of the latest registered schema as a read-only table, with their types, whether they are required, and their own descriptions on hover; the header shows the schema type, the retention and the maximum message size. The schema itself is shown and changed in Edit.
  • Used by - the nodes of non-archived scenarios that name the topic, whether they read or write it, and the scenario's status.
  • Messages - up to 30 most recent messages on the topic, newest first, as a table: the message time, its key and a column per field of the schema (fields the schema does not name follow). Click a row to see every field in full with where the message came from (partition, offset, schema ids, headers), and Copy as JSON to copy its value; JSON shows the whole sample as JSON and Copy all copies it. Avro messages are shown the same way as JSON ones: union values without their branch names, decimals as numbers, dates and times as ISO text (2026-11-08T13:45:00.123Z). An integer too long to be shown exactly (above 2^53, like a 19-digit id) is shown and copied as text with all its digits. A message key in Avro is shown as it comes, since the page reads only the schema of the message value. Messages are read only when you open this tab, refreshed every 10 seconds while it stays open and no row is opened, and on Refresh. The tab needs the Topic.ViewMessages permission.

and the actions:

  • Edit - opens the Edit topic dialog with the same fields as when adding; the name cannot be changed. Changing the schema registers it as a new schema version, after checking it against the topic's compatibility; a schema that breaks it is refused and nothing changes, neither the schema nor the retention. Choosing No schema removes the topic's schema. Changing the retention does not change the topic's segment time. To make a shorter retention take effect sooner on an existing topic, a Kafka administrator sets segment.ms on it. A topic created with a short retention keeps its short segment time when you lengthen the retention, so it piles up many small segments, each a few files on every broker that holds it; a Kafka administrator removes segment.ms from it to go back to the cluster's segment time. Confirm with Update topic.
  • Clone - opens the Add new topic dialog prefilled with this topic's schema and retention, with an empty name, so you can create a similar topic under a new name.
  • Delete - deletes the topic together with its schema after you confirm with Delete topic. Messages on the topic are lost. While an active scenario uses the topic, Delete is greyed out and its tooltip names the scenarios to stop first: a scenario counts as active in any status other than NOT_DEPLOYED, CANCELED and FINISHED, also PROBLEM. Scenarios that are not running and fragments do not block it; the confirmation lists those scenarios, because they cannot be deployed until the topic exists again. Only the scenarios you can see are checked, and only nodes that name the topic directly, as in Used by.

Schema compatibility​

Scenarios that read a topic were validated against its schema, and the topic still holds messages written with earlier versions of it. The compatibility of a topic tells the schema registry which changes of the schema to accept, so a new version does not break them:

CompatibilityA new schema version is accepted when
Backwardit reads messages written with the previous version - for example it adds a field with a default value or removes a field. A scenario updated to the new schema still reads the older messages on the topic.
Backward transitiveit reads messages written with any earlier version.
Forwardthe previous version reads messages written with it - for example it adds a field or removes a field that has a default value. Scenarios not updated yet still read the new messages.
Forward transitiveevery earlier version reads messages written with it.
Fullboth backward and forward, against the previous version.
Full transitiveboth backward and forward, against every earlier version.
Nonealways: any change is accepted, also one that the scenarios reading the topic cannot follow.

A refused schema comes back with what breaks the compatibility, for example a field that was added without a default value. To register such a change anyway, choose a weaker compatibility in the same dialog. Saving with None registers any schema; the schema registry keeps the topic's compatibility as chosen for the next versions.

A JSON Schema object allows properties it does not list unless it sets "additionalProperties": false. Under Backward the schema registry therefore refuses a property added to such an object, as an older message may already have had it with another type. Closing the object in the new version does not help, as the registry refuses that change too: choose Forward for this change. An object that sets "additionalProperties": false from its first version accepts added optional properties under Backward.

A topic whose schema was saved on the Topics page before Nussknacker 2.3 has the compatibility None, which the page set back then. Edit it to choose another one.

Limits and errors​

  • A topic name that is empty, too long or contains other characters is rejected (Invalid topic name).
  • Retention must be at least 1 hour.
  • Adding a topic whose name is taken fails with Topic [<name>] already exists.
  • An installation can limit how many topics it holds; once it is reached, adding fails with Topic limit of <n> reached for this instance.
  • A schema can be registered only when the Kafka configuration has a schema registry; otherwise saving a topic with a schema fails with No schema registry is configured for this instance.
  • An operation the cluster's credentials do not allow fails with Kafka [<cluster>] refused this with the credentials Nussknacker uses for it: <reason>.
  • A schema that breaks the topic's compatibility fails with The schema is not <compatibility> compatible with the previous version of the topic's schema, followed by what breaks it.

Creating a topic from a node​

The Topic name parameter of Kafka sources and sinks (including the Topic source and sink) has a Create New Topic button next to it while you edit the node, whenever the Topics extension is enabled. It creates a topic with a random name, no schema and the default retention, and puts its name into the parameter. Open the Topics page afterwards to add a schema or rename it by cloning.

AI assistant​

When the Topics extension is enabled, the Designer AI assistant can list topics, read a topic with its schema, preview its last messages, create a topic and update its retention or schema. It cannot delete topics.

Configuration​

This section is for administrators. The Topics page consists of the Topics extension of the Designer (the /api/custom/topics API) and a tab in the main menu configuration:

tabs: ${tabs} [
{id: "topics", url: "nuEnterpriseTopics/tab@/static/topics/remoteEntry.js", title: "Topics", type: "Remote"}
]

The Create New Topic button and the AI assistant tools are available only when this tab is configured.

Every endpoint of /api/custom/topics takes an optional cluster query parameter with an id from GET /api/custom/topics/-/clusters; without it the request goes to the dedicated Kafka, so the Create New Topic button, the AI assistant and nu-cli keep working on it. The other clusters are the Kafka component providers of the same model configuration (components.<id> with providerType: "kafka") whose bootstrap.servers differ from the dedicated one's; a Kafka added under Integrations is one of them.

The extension is configured in the Designer configuration under topicsExtension:

topicsExtension {
processingType: "streaming"
topicLimit: 100
defaultPartitions: 1
defaultReplicationFactor: 1
defaultRetentionHours: 24
maxMessageBytes: 102400
}
Parameter nameTypeDefault valueDescription
topicsExtension.processingTypestringProcessing type (scenario type) whose model configuration supplies the Kafka and schema registry connection, read from components.webhook.config (the webhook-kafka component provider), and the namespace of topic names
topicsExtension.topicLimitintno limitOptional. Maximum number of topics of this instance; adding a topic beyond it is refused
topicsExtension.defaultPartitionsintNumber of partitions of a newly created topic
topicsExtension.defaultReplicationFactorshortReplication factor of a newly created topic
topicsExtension.defaultRetentionHoursintRetention (retention.ms) of a new topic when none is given, e.g. by the Create New Topic button
topicsExtension.maxMessageByteslongmax.message.bytes of a newly created topic - the size of the biggest message it accepts

All parameters except topicLimit are required. When a namespace is configured, topic names are prefixed with it in Kafka, the page shows them without the prefix, and only topics with this instance's prefix are listed. A schema is registered under the <topic>-value subject, and only when schema.registry.url is set in the Kafka properties.

To give a topic with a short retention a matching segment time, the extension reads the cluster's segment time (log.roll.ms or log.roll.hours) from a broker. This needs the DescribeConfigs permission on the Kafka cluster resource, which an account limited to topic permissions lacks. Without it, the extension assumes Kafka's default of 7 days.