front matter metadata for LLM

componentName: Throttle processingModes: [Streaming] componentPaletteGroup: Flow

Throttle

Description

Limits the emission rate of events per group using a token bucket. For each group (defined by Key by), a bucket of Burst capacity tokens refills at one token every Time window / Number of events. An event is emitted immediately if a token is available, otherwise it is buffered and released when the next token refills. Events are never dropped and never reordered within a group. After an idle period up to Burst capacity events are released at once; the sustained rate stays at most Number of events per Time window. The default is 100 events per 1 second per group, burst 100.

Parameters and configuration

Name	Description
Key by	Expression assigning each event to a group; events sharing a key are throttled independently. Example: #input.customerId. Leave as '' to throttle the whole stream as one group. Events whose key is null are reported as errors and not emitted.
Number of events	Maximum number of events emitted per Time window, per group.
Time window	Window over which Number of events events are released per group. Effective spacing is Time window / Number of events.
Burst capacity (advanced)	Maximum number of events released instantly after an idle period. Empty = Number of events. 1 = no burst (even spacing). Greater than Number of events allows larger spikes at the same average rate.
Time mode (advanced)	Processing time (wall clock, default) or Event time.

How it works

1. Each event is grouped by Key by.
2. The component schedules each event at max(now, TAT - tau), where TAT is the per-group theoretical arrival time and tau = (Burst capacity - 1) * (Time window / Number of events). This releases up to Burst capacity events immediately after idle, then paces the rest.
3. When the scenario input ends (a bounded source finishing, or stop with drain), any still-buffered events are flushed so they are not lost. On a processing-time restart, each group's backlog is re-paced from the restart time rather than dumped.

Limitations

• No back-pressure, unbounded buffering. If a group's input rate sustainedly exceeds its throttle rate, its buffer grows without bound. The component delays events; it does not drop them or slow the source.
• Per-group state grows with key cardinality. The per-group scheduling state is not cleaned up, so it grows with the number of distinct keys seen and does not shrink for idle keys.
• Effective rate is capped at ~1000 events/s per group by millisecond timer granularity. Configurations requiring a sub-millisecond spacing are rejected at validation.

Metrics

• throttle.bufferedEvents — number of events currently buffered (per node).
• throttle.delay — histogram of per-event scheduled delay, in milliseconds.

Examples

To emit at most 100 events per second per customer:

• Key by: #input.customerId
• Number of events: 100
• Time window: 1 second

Each customer's events are released at up to 100/s; bursts up to 100 pass immediately after a lull.