Skip to main content
Version: Current

Scenario log

The scenario log is a journal of each deployment of a scenario, kept in Designer's database: the operations run on it, its status changes, the periods when it was not fine and the exceptions it threw. It is not a text log: what loggers write, also the Log nodes of the scenario, stays in the logs of Designer and the engines.

The self-managed installation bundle enables the scenario log and the Deployments panel by default. Other installations can enable them in Designer configuration and restart Designer:

scenarioLog.enabled: true

Set scenarioLog.enabled: false or SCENARIO_LOG_ENABLED=false to disable them in the installation bundle.

With only deployment diagnostics on (deploymentDiagnostics.enabled), Designer keeps part of the log: the exceptions, and each operation Designer runs with its outcome, the causes of its failure and its deployments (runs of scheduled scenarios not) - which deployment diagnostics show for the actions they list - without phases, lines, episodes or the panel. The assistant and the API read that part. Turning the scenario log on or off changes nothing about how a Deployment Manager deploys. With both features off, live-data collectors still aggregate and upload bounded exception batches. Designer acknowledges and discards these batches without persisting exception groups. The flags control persistence, background diagnostic reads and presentation; they do not disable runtime exception collection.

Polling runs every five seconds while an operation or episode is active and every thirty seconds otherwise. Readers can send up to 200 entry revisions in knownEntries to avoid receiving unchanged entries again in the cursor overlap window. Exception samples load separately when a group is expanded.

The panel​

The Deployments panel of the scenario view shows the latest 5 deployments as one log, newest first, each under a line with its date, version, author and state. A deployment is one run of a version: the same version deployed again is another deployment. The selector at the top switches to the latest 100, or to one deployment in full; it also lists deploys that failed before they started, runs of a scheduled scenario and, under Not tied to a deployment, operations that concerned none, such as a cancel when nothing ran. The log refreshes on its own.

Operations​

Every deploy, redeploy, cancel, stop, savepoint, scheduled run and recovery after a restart of Designer or the engine is an operation, shown as a checklist of phases: done, failed, running with its progress (an amount, a number of checks or time against a timeout), planned, or skipped with the reason. A phase that ends shows its result, such as the size of the uploaded model or the savepoint taken. When a scenario type has several deployment targets, each phase names its target.

Warnings and errors are shown at once. Expanding an operation loads all its lines, including every request it sent to the engine (the Flink REST API, Kubernetes, Ververica); requests repeated while polling are one line with their count.

A deploy over another version says what it changes: nodes changed, added or removed, the scenario's properties, and for a request-response scenario whether its API changed. A redeploy is shown in full under the deployment it created, and as one line under the deployment it replaced. A failed operation shows its error and the classified causes, which deployment diagnostics and the assistant read too.

On standalone Flink a deploy ends when the job leaves DURING_DEPLOY (waitForDuringDeployFinish), as without the scenario log. On the Flink K8s operator and Lite on Kubernetes a deploy ends once the engine accepted it, and a startup operation follows it until the job or the pods run; it fails when they fail or do not run in time. An error the engine reports while starting and then clears by itself, such as a volume not mounted yet, is a warning only when it is still reported a minute later.

States and episodes​

Each status the engine reports for the deployment is a line; a change that repeats one of the last two within 10 minutes, as in a restart loop, is that line again with its count. A period when the deployment is not fine is an episode, one line updated while it lasts:

EpisodeStarts when
Restartingthe engine reports the deployment restarting
Problemthe engine reports a problem
Unknownthe engine or the deployment target could not be asked
Deploy slowthe deployment is not running 5 minutes after the engine accepted it
Checkpoint stalea running Flink job that takes checkpoints has completed none for 10 minutes

It ends when the deployment runs again, or is canceled or finished. Meanwhile Designer reads the Deployment Manager's diagnostics, every 30 seconds at first and up to every 5 minutes later, and keeps what changed as evidence: the job's status, each failure with its cause and node, workers that left. An unknown episode is not read: the engine could not be asked, and while a deployment target cannot be read, the other episodes on it wait too. The Health page lists the open episodes.

scenarioLog {
episodes {
deepReadInterval: 30s
deepReadMaxInterval: 5m
deploySlowAfter: 5m
}
thresholds {
checkpointStaleAfter: 10m # a running job that completed no checkpoint for this long
withoutCheckpointsRecheckAfter: 1h # a job whose engine reports no checkpoints is checked again only after
}
}

Exceptions​

Every exception a running scenario throws is kept, grouped by node (or outside any node), exception class and cause: processing.timeout, processing.connection_failed, processing.secure_connection_failed or processing.failed, as in the technical log. A group is one line: where it happened, how often, a bar per minute of the last hour, and whether it is ongoing, resolved (none for 5 minutes) or muted. Expanding it shows the message and samples with their cause chains and stacks: the first exception of the group and the latest 3, a new sample taken at most once a minute. A group that started during an operation, or up to 2 minutes after it, names that operation.

A Flink scenario that fails before its records flow is a group too, with its phase in place of processing: compilation for the scenario's compilation, one group per node and one for errors of the whole scenario; startup for loading the model, its classes included, and opening the components - an enricher whose client cannot connect is startup.connection_failed; execution for the job failing to be built or submitted. The job sends it at once, waiting at most 3 seconds, and then fails as it would have. It needs the scenario deployed with this version.

Not every failure gets there: the sources and sinks that connect when the job starts, such as Kafka's, report no group, and a job that fails after it was submitted, or a JVM that dies abruptly, sends nothing. Such a failure is a restarting or problem episode, with the engine's evidence where the engine provides it - standalone Flink and Ververica do, the Flink K8s operator does not yet.

Exceptions travel with the live data uploads, so they are collected where live data is enabled (liveDataPreview.enabled), whether or not anyone watches, also for 5 minutes after a deployment failed. The variables of a sample are kept only when the model allows it:

modelConfig {
scenarioLog.captureVariables: true # off by default
}

Anyone who may deploy the scenario can mute a group: it stays in the log, but no longer counts as a problem.

Phases​

Designer runs designer.prepare, designer.validate and designer.validateOnEngine (the Deployment Manager's checks, such as free slots) before the phases of the engine:

CodePhase
flink.savepointFlink: savepoint of the job a redeploy replaces, against the timeout
flink.stateCheckthe savepoint's state fits the new version (scenarioStateVerification)
flink.cancelcancelling the replaced job, or the job of a cancel
flink.slotsfree task slots: needed and free
flink.jarBuildbuilding the model jar, or reusing the one built before
flink.jarUploaduploading the model jar, unless Flink has it already
flink.runJobrunning the job
flink.waitForStartwaiting for the job to leave DURING_DEPLOY
flink.stopstopping the job with a savepoint
operator.jarTransferFlink K8s operator: copying the model jar to the pod or shared directory
operator.applycreating or updating the FlinkDeployment
operator.waitForStartstartup: until the job runs (waitForStart, 10 minutes)
operator.deletedeleting the FlinkDeployment
operator.waitForRemovaluntil it is removed (cancelTimeout, 5 minutes)
k8s.quotaCheckLite on Kubernetes: resource quotas, and the slug is free
k8s.applycreating or updating one Kubernetes object
k8s.cleanupremoving ConfigMaps, Secrets and Services of previous versions
k8s.waitForReadystartup: until the pods are ready, or why not (waitForReady, 5 minutes)
k8s.deletedeleting the scenario's objects on cancel
embedded.stopPreviousLite embedded: stopping the version running before
embedded.start, embedded.stopstarting or stopping the scenario
vvp.sharedModelCopyVerverica: writing the model to sharedModelDirectory
vvp.artifactUploaduploading the model jar
vvp.descriptorUploaduploading the execution descriptor
vvp.stopWithSavepointstopping the previous job with a savepoint
vvp.deploymentApplycreating or updating the Ververica deployment
vvp.startJobstarting the job, with the restore strategy
vvp.cancel, vvp.stop, vvp.savepointcancelling, stopping or taking a savepoint of the job

For the assistant and nu-cli​

The assistant and nu-cli mcp read the log with two tools. get_scenario_log gives the current deployment and the 2 before it, or one chosen deployment, and compares their exception groups. get_scenario_log_entry gives one operation with its phases, or one exception group with its samples. While someone has the scenario open, a deployment that starts restarting, gets a problem or stops completing checkpoints, and a new exception group, show as notifications.

Several Designer instances​

Each instance records the operations it runs and marks them alive every minute; an operation of an instance that stopped is abandoned 5 minutes after its last mark. Status changes and episodes are recorded by the leader, which takes over the open episodes when it becomes the leader.

Access and retention​

Anyone who can view the scenario can read its log. Error texts, ERROR lines, episodes' evidence and exceptions' messages and samples, which may carry the data of processed records, are shown only to users who may view the scenario's live data, and to the assistant only with aiDataAccess.allowRuntimeData.

The log keeps what concerns the latest 100 deployments of a scenario and those still running, for at most 180 days:

scenarioLog {
retention {
maxAge: 180d
deploymentsPerScenario: 100
collectorsSilentFor: 1d # how long the registration of a job's exception batches is kept after its last one
}
}