Topic (Source)
Description
Topic Source allows to read events from the Kafka topic hosted Nussknacker Cloud ("internal topics"). You can use internal Kafka topics to:
- route data between scenarios
- send events to a topic via public HTTPS or WebSocket endpoint
- receive results via HTTP callback or WebSocket subscription
Use the Topic link (top right of the Designer window) to create a topic; read this document to learn how to create a Nussknacker Cloud hosted Kafka topic.
Parameters and configuration
| Name | Description |
|---|---|
| Topic name | Kafka topic to write to |
| Version | Schema version |
Advanced parameters
| Name | Description |
|---|---|
| Event time | Expression which evaluates to the time when the event was created. For Kafka sources, creation timestamp is available in the Kafka event and can be accessed in SpEL as #inputMeta.timestamp. Check here |
| Max out-of-orderness | The maximum amount of time an element is allowed to be late before being ignored when computing the result for time-based stream transformations: aggregates in time windows and joins. To read more about this mechanism see Flink documentation |
| Idleness | The time period after which partition is marked as idle if no events are received from it. To read more about this mechanism see Flink documentation. |
Additional considerations
Event time
The Kafka event timestamp is the default value suggested by Designer for the Event time field. If you do not use this default, read here to learn more about Instant data type. If provided as a numeric value rather than Instant, it must be in epoch milliseconds (not seconds); a Long is the most natural data type for representing this UTC millisecond value.
Getting event time wrong can lead to subtle errors in time-based logic that are difficult to diagnose - if such logic behaves unexpectedly, this should be the first thing to verify.