Type Members

1.  case class InputMeta[K](key: K, topic: String, partition: Integer, offset: Long, timestamp: Long, timestampType: TimestampType, headers: Map[String, String], leaderEpoch: Integer) extends Product with Serializable

   InputMeta represents kafka event metadata.

   InputMeta represents kafka event metadata. It is based on org.apache.kafka.clients.consumer.ConsumerRecord. Ignored fields: checksum, serializedKeySize, serializedValueSize.

   K

   - type of event key

   key

   - event key

   topic

   - kafka topic

   partition

   - kafka partition

   offset

   - event offset

   timestamp

   - event timestamp

   timestampType

   - see org.apache.kafka.common.record.TimestampType

   headers

   - event headers converted to map

   leaderEpoch

   - number of leaders previously assigned by the controller (> 0 indicates leader failure)

2.  class InputMetaToJson extends ToJsonEncoder
3.  class KafkaContextInitializer[K, V] extends BasicContextInitializer[ConsumerRecord[K, V]]

   KafkaContextInitializer initializes Context variables with data provided in raw kafka event (see org.apache.kafka.clients.consumer.ConsumerRecord).

   KafkaContextInitializer initializes Context variables with data provided in raw kafka event (see org.apache.kafka.clients.consumer.ConsumerRecord). It is used when flink source function produces stream of ConsumerRecord (deserialized to proper key-value data types). Produces pl.touk.nussknacker.engine.api.Context with two variables: - default "input" variable which is set up with ConsumerRecord.value - metadata of kafka event, see InputMeta

   K

   - type of key of deserialized ConsumerRecord

   V

   - type of value of deserialized ConsumerRecord

4.  class KafkaSourceFactory[K, V] extends SourceFactory with SingleInputGenericNodeTransformation[Source] with WithCachedTopicsExistenceValidator with WithExplicitTypesToExtract

   Base factory for Kafka sources with additional metadata variable.

   Base factory for Kafka sources with additional metadata variable. It is based on pl.touk.nussknacker.engine.api.context.transformation.SingleInputGenericNodeTransformation that allows custom ValidationContext and Context transformations, which are provided by KafkaContextInitializer Can be used for single- or multi- topic sources (as csv, see topicNameSeparator and extractTopics).

   Features:

   • fetch latest N records which can be later used to test process in UI Fetching data is defined in source which may extends pl.touk.nussknacker.engine.api.process.TestDataGenerator. See pl.touk.nussknacker.engine.kafka.KafkaUtils#readLastMessages
   • reset Kafka's offset to latest value - forceLatestRead property, see pl.touk.nussknacker.engine.kafka.KafkaUtils#setOffsetToLatest

   K

   - type of key of kafka event that is generated by raw source (SourceFunction).

   V

   - type of value of kafka event that is generated by raw source (SourceFunction).