Window Top-N

Description

The Window Top-N node (window-top-n) splits the stream into fixed (tumbling) windows of event time and emits, for every window and every key, the N records with the highest (or lowest) value of a chosen field - e.g. the three largest transactions of every customer in every hour.

Under the hood the node runs a Flink SQL window Top-N query over the incoming records.

• The input of the node is one record variable available before the node (by default #input) - all its fields, or a selected subset, become the fields of the output, plus record_time, window_start and window_end.
• The node clears all upstream variables - after the node only the output variable is available.

The node is available only in the Streaming processing mode, for unbounded streams.

Parameters and configuration

Name	Description
Input	The record variable and the fields to keep in the output.
Partition by	The fields making up the key; the ranking is computed separately for every combination of their values. With no fields, one ranking per window.
Order by	The field the records are ranked by, and the direction: DESC keeps the highest values, ASC the lowest.
N	How many records per window and key to keep, at least 1.
Window size	The length of the window, e.g. 1 HOUR.
Output variable name	The name of the variable holding a kept record.

Additional considerations

• The node is stateful: open windows live in Flink state and are restored from a savepoint when the scenario is redeployed. Changing the key, the order or the window size may make the state incompatible.
• Records are emitted when the window closes, not when they arrive. Late records (behind the watermark) are dropped. Records with equal values of the order field are ranked in an unspecified order.
• The node can't be mocked in scenario tests - it has to be fed real records.