Scenario log
The scenario log is a journal of each deployment of a scenario, kept in Designer's database: the operations run on it, its status changes, the periods when it was not fine and the exceptions it threw. It is not a text log: what loggers write, also the Log nodes of the scenario, stays in the logs of Designer and the engines.
The self-managed installation bundle enables the scenario log and the Deployments panel by default. Other installations can enable them in Designer configuration and restart Designer:
scenarioLog.enabled: true
Set scenarioLog.enabled: false or SCENARIO_LOG_ENABLED=false to disable them in the installation bundle.
With only deployment diagnostics on (deploymentDiagnostics.enabled), Designer keeps part of the log: the exceptions,
and each operation Designer runs with its outcome, the causes of its failure and its deployments (runs of scheduled
scenarios not) - which deployment diagnostics show
for the actions they list - without phases, lines, episodes or the panel. The assistant and the API read that part.
Turning the scenario log on or off changes nothing about how a Deployment Manager deploys.
With both features off, live-data collectors still aggregate and upload bounded exception batches. Designer
acknowledges and discards these batches without persisting exception groups. The flags control persistence,
background diagnostic reads and presentation; they do not disable runtime exception collection.
Polling runs every five seconds while an operation or episode is active and every thirty seconds otherwise.
Readers can send up to 200 entry revisions in knownEntries to avoid receiving unchanged entries again in the
cursor overlap window. Exception samples load separately when a group is expanded.
The panel
The Deployments panel of the scenario view shows the latest 5 deployments as one log, newest first, each under a line with its date, version, author and state. A deployment is one run of a version: the same version deployed again is another deployment. The selector at the top switches to the latest 100, or to one deployment in full; it also lists deploys that failed before they started, runs of a scheduled scenario and, under Not tied to a deployment, operations that concerned none, such as a cancel when nothing ran. The log refreshes on its own.
Operations
Every deploy, redeploy, cancel, stop, savepoint, scheduled run and recovery after a restart of Designer or the engine is an operation, shown as a checklist of phases: done, failed, running with its progress (an amount, a number of checks or time against a timeout), planned, or skipped with the reason. A phase that ends shows its result, such as the size of the uploaded model or the savepoint taken. When a scenario type has several deployment targets, each phase names its target.
Warnings and errors are shown at once. Expanding an operation loads all its lines, including every request it sent to the engine (the Flink REST API, Kubernetes, Ververica); requests repeated while polling are one line with their count.
A deploy over another version says what it changes: nodes changed, added or removed, the scenario's properties, and for a request-response scenario whether its API changed. A redeploy is shown in full under the deployment it created, and as one line under the deployment it replaced. A failed operation shows its error and the classified causes, which deployment diagnostics and the assistant read too.
On standalone Flink a deploy ends when the job leaves DURING_DEPLOY (waitForDuringDeployFinish), as without the
scenario log. On the Flink K8s operator and Lite on Kubernetes a deploy ends once the engine accepted it, and a
startup operation follows it until the job or the pods run; it fails when they fail or do not run in time. An error
the engine reports while starting and then clears by itself, such as a volume not mounted yet, is a warning only when
it is still reported a minute later.
States and episodes
Each status the engine reports for the deployment is a line; a change that repeats one of the last two within 10 minutes, as in a restart loop, is that line again with its count. A period when the deployment is not fine is an episode, one line updated while it lasts:
| Episode | Starts when |
|---|---|
| Restarting | the engine reports the deployment restarting |
| Problem | the engine reports a problem |
| Unknown | the engine or the deployment target could not be asked |
| Deploy slow | the deployment is not running 5 minutes after the engine accepted it |
| Checkpoint stale | a running Flink job that takes checkpoints has completed none for 10 minutes |
It ends when the deployment runs again, or is canceled or finished. Meanwhile Designer reads the Deployment Manager's diagnostics, every 30 seconds at first and up to every 5 minutes later, and keeps what changed as evidence: the job's status, each failure with its cause and node, workers that left. An unknown episode is not read: the engine could not be asked, and while a deployment target cannot be read, the other episodes on it wait too. The Health page lists the open episodes.
scenarioLog {
episodes {
deepReadInterval: 30s
deepReadMaxInterval: 5m
deploySlowAfter: 5m
}
thresholds {
checkpointStaleAfter: 10m # a running job that completed no checkpoint for this long
withoutCheckpointsRecheckAfter: 1h # a job whose engine reports no checkpoints is checked again only after
}
}
Exceptions
Every exception a running scenario throws is kept, grouped by node (or outside any node), exception class and cause:
processing.timeout, processing.connection_failed, processing.secure_connection_failed or processing.failed, as
in the technical log. A group is one line: where it happened, how often, a bar per minute of the last hour, and
whether it is ongoing, resolved (none for 5 minutes) or muted. Expanding it shows the message and samples with their
cause chains and stacks: the first exception of the group and the latest 3, a new sample taken at most once a minute. A group that started during an operation, or up to 2 minutes after it, names that operation.
A Flink scenario that fails before its records flow is a group too, with its phase in place of processing:
compilation for the scenario's compilation, one group per node and one for errors of the whole scenario;
startup for loading the model, its classes included, and opening the components - an enricher whose client cannot
connect is startup.connection_failed; execution for the job failing to be built or submitted. The job sends it at
once, waiting at most 3 seconds, and then fails as it would have. It needs the scenario deployed with this version.
Not every failure gets there: the sources and sinks that connect when the job starts, such as Kafka's, report no group, and a job that fails after it was submitted, or a JVM that dies abruptly, sends nothing. Such a failure is a restarting or problem episode, with the engine's evidence where the engine provides it - standalone Flink and Ververica do, the Flink K8s operator does not yet.
Exceptions travel with the live data uploads, so they are collected where live data is enabled
(liveDataPreview.enabled), whether or not anyone watches, also for 5 minutes after a deployment failed. The
variables of a sample are kept only when the model allows it:
modelConfig {
scenarioLog.captureVariables: true # off by default
}
Anyone who may deploy the scenario can mute a group: it stays in the log, but no longer counts as a problem.
Phases
Designer runs designer.prepare, designer.validate and designer.validateOnEngine (the Deployment Manager's
checks, such as free slots) before the phases of the engine:
| Code | Phase |
|---|---|
flink.savepoint | Flink: savepoint of the job a redeploy replaces, against the timeout |
flink.stateCheck | the savepoint's state fits the new version (scenarioStateVerification) |
flink.cancel | cancelling the replaced job, or the job of a cancel |
flink.slots | free task slots: needed and free |
flink.jarBuild | building the model jar, or reusing the one built before |
flink.jarUpload | uploading the model jar, unless Flink has it already |
flink.runJob | running the job |
flink.waitForStart | waiting for the job to leave DURING_DEPLOY |
flink.stop | stopping the job with a savepoint |
operator.jarTransfer | Flink K8s operator: copying the model jar to the pod or shared directory |
operator.apply | creating or updating the FlinkDeployment |
operator.waitForStart | startup: until the job runs (waitForStart, 10 minutes) |
operator.delete | deleting the FlinkDeployment |
operator.waitForRemoval | until it is removed (cancelTimeout, 5 minutes) |
k8s.quotaCheck | Lite on Kubernetes: resource quotas, and the slug is free |
k8s.apply | creating or updating one Kubernetes object |
k8s.cleanup | removing ConfigMaps, Secrets and Services of previous versions |
k8s.waitForReady | startup: until the pods are ready, or why not (waitForReady, 5 minutes) |
k8s.delete | deleting the scenario's objects on cancel |
embedded.stopPrevious | Lite embedded: stopping the version running before |
embedded.start, embedded.stop | starting or stopping the scenario |
vvp.sharedModelCopy | Ververica: writing the model to sharedModelDirectory |
vvp.artifactUpload | uploading the model jar |
vvp.descriptorUpload | uploading the execution descriptor |
vvp.stopWithSavepoint | stopping the previous job with a savepoint |
vvp.deploymentApply | creating or updating the Ververica deployment |
vvp.startJob | starting the job, with the restore strategy |
vvp.cancel, vvp.stop, vvp.savepoint | cancelling, stopping or taking a savepoint of the job |
For the assistant and nu-cli
The assistant and nu-cli mcp read the log with two tools. get_scenario_log gives the current deployment and the 2
before it, or one chosen deployment, and compares their exception groups. get_scenario_log_entry gives one
operation with its phases, or one exception group with its samples. While someone has the scenario open, a
deployment that starts restarting, gets a problem or stops completing checkpoints, and a new exception group, show as
notifications.
Several Designer instances
Each instance records the operations it runs and marks them alive every minute; an operation of an instance that stopped is abandoned 5 minutes after its last mark. Status changes and episodes are recorded by the leader, which takes over the open episodes when it becomes the leader.
Access and retention
Anyone who can view the scenario can read its log. Error texts, ERROR lines, episodes' evidence and exceptions'
messages and samples, which may carry the data of processed records, are shown only to users who may view the
scenario's live data, and to the assistant only with aiDataAccess.allowRuntimeData.
The log keeps what concerns the latest 100 deployments of a scenario and those still running, for at most 180 days:
scenarioLog {
retention {
maxAge: 180d
deploymentsPerScenario: 100
collectorsSilentFor: 1d # how long the registration of a job's exception batches is kept after its last one
}
}