Skip to main content
Version: Current

Model configuration

Model definition is part of a scenario type definition. There can be multiple scenario types in one Nussknacker installation, consequently there will also be multiple model definitions in such a case. Check configuration areas to understand where Model configuration should be placed in the Nussknacker configuration. If you deploy to K8s using Nussknacker Helm chart, check here how to supply additional model configuration.

Model defines how to configure components and certain runtime behavior (e.g. error handling) for a given scenario type. Model configuration is processed not only at the Designer but also passed to the execution engine (e.g. Flink), that’s why it’s parsed and processed a bit differently:

  • Some Components can use a special mechanism which resolves and adds additional configuration during deployment, which is then passed to the execution engine. Such configuration is read and resolved only at the Designer. Example: OpenAPI enrichers need to read its definition from external sites - so e.g. Flink cluster does not have to have access to the site with the definition.
  • There is additional set of defaults, taken from defaultModelConfig.conf if it exists on the classpath. The standard Nussknacker installation uses the one from here, installations using certain code customizations may use a different one.

ClassPath configuration​

Nussknacker looks for components and various extensions in jars on the Model classpath, default config example here to see where classpath can be configured.

By default, in case of Flink streaming scenario type, the following configuration is used:

classPath: ["model/defaultModel.jar", "model/flinkExecutor.jar", "components/flink", "components/common", "flink-dropwizard-metrics-deps/"]

Make sure you have all necessary entries properly configured:

  • Jar with model - unless you used custom model, this should be model/defaultModel.jar
  • All jars with additional components, e.g. "components/flink/flinkBase.jar", "components/flink/flinkKafka.jar"
  • flinkExecutor.jar for Flink engine. This contains executor of scenarios in Flink cluster.

Note that as classPath elements you can use:

If the given path element in the classPath is relative, it should be relative to the path determined by the $WORKING_DIR environment variable.

By default, all classpath entries are packed into the artifact that the deployment manager sends to the engine on every deployment. For the Flink engine, entries that the cluster already provides (e.g. baked into the Flink image) can be excluded — see modelArtifact.

Some JAR files should not be listed here at all. Everything on the classPath is loaded by the model classloader, which is created anew whenever the model is reloaded, so a library that starts threads it never stops - the Oracle JDBC driver, for example - leaks a classloader on every reload, in the Designer (its MiniCluster included) as much as on the cluster. Put such a file in the Designer's lib directory, where the application classloader loads it once and model code still reaches it, and in Flink's lib directory - see CLASSPATH and the driver setup described in SQL.

Components configuration​

Nussknacker comes with a set of provided components. Some of them (e.g. filter, variable, aggregations in Flink, for-each, union) are predefined and accessible by default. Others need additional configuration - the most important ones are enrichers, where you have to set e.g. JDBC URL or external service address.

Check Integration documentation for the details on how to configure the following components:

  • OpenAPI Supports accessing external APIs directly from scenario
  • SQL Supports access to SQL database engines
  • Machine Learning Infers ML models

Configuration of component providers​

Below you can see typical component configuration, each section describes configuration of one component provider.

  components {
    sqlHsql {
      providerType: databaseEnricher
      jdbcUrl: jdbc:hsql...//
      admin/pass
    }
    sqlOracle {
      providerType: databaseEnricher
      jdbcUrl: jdbc:oracle...//
      admin/pass
    }
    prinzH20 {
      providerType: prinzH20
      h2oLocation:
    }
    prinzMlFlow {
      #this is not needed, as configuration is named just as provider
      #providerType: prinzMLFlow
      mlFlowUrl:
    }
    #we can disable particular component provider, if it's not needed in our installation
    #note: you cannot disable certain basic components like filter, variable, choice and split
    aggregation {
      disabled: true
    }
  }

Common configuration options that area available for component providers:

  • providerType - type of provider
  • disabled - allow to disable given component provider (default false)
  • componentPrefix - prefix that will be added to components provided by provider (default empty string)

Configuration of UI attributes of components​

In model configuration you can also define some UI attributes of components. This can be useful for tweaking of appearance of generated components (like from OpenAPI), in most cases you should not need to defined these settings. The settings you can configure include:

  • icons - icon property
  • documentation - docsUrl property
  • label - label property
  • should component be disabled - disabled property
  • in which toolbox panel the component should appear (componentGroup property) - see component group mapping
  • params configuration (allows to override default component settings):
    • editor - BoolParameterEditor, SpelTemplateParameterEditor, DateParameterEditor etc.
    • validators - MandatoryExpressionValidator, NotBlankValidator, RegExpValidator
    • defaultValue
    • label
    • hintText

Example (see dev application config for more examples):

  componentsUiConfig {
    customerService {
      params {
        serviceIdParameter {
            defaultValue: "customerId-10"
            editor: "SpelTemplateParameterEditor"
            validators: [ 
              {
                type: "RegExpValidator"
                pattern: "customerId-[0-9]*"
                message: "has to match customer id format"
                description: "really has to match..."
              }
            ]
            label: "Customer id (from CRM!)"
            hintText: "Input customerID in proper format"
        }
      }
      docsUrl: "https://en.wikipedia.org/wiki/Customer_service"
      icon: "icon_file.svg"
      label: "Customer Service"
    }
  }

As a key in the configuration you can use component name or full component identifier which is in the componentType-componentName format. The second approach is mostly useful when there is more than one component with the same name (e.g. source-kafka and sink-kafka components). Available component types:

  • source - for components located in the sources toolbox panel
  • sink - for components located in the sinks toolbox panel
  • service - for components located in the enrichers and services toolbox panels
  • custom - for components located in the custom and optionalEndingCustom toolbox panels
  • built-in - for built-in components (choice, filter, record-variable, split, variable) located in the base toolbox panel

You can add additional links that will be shown in Components tab. They can be used e.g. to point to custom dashboards with component performance or point to some external system (link to documentation is configured by default). The config format is as follows:

componentLinks: [
  {
    id: "sourceSystem"
    title: "Source system"
    icon: "/assets/components/CustomNode.svg"
    url: "https://myCustom.com/dataSource/$componentName" 
    supportedComponentTypes: ["service"]
    requiredPermission: "ServicePermission"
  }
]

Fields title, icon, url can contain templates: $componentId nad $componentName which are replaced by component data. Param supportedComponentTypes means component's types which can support links. Optional param requiredPermission allows link access to users with a given permission.

Component group mapping​

Components are assigned to component groups, which are rendered as toolbox panels. Out of the box Nussknacker displays the built-in groups under readable names, so no configuration is needed to get a properly named toolbox. The default names are equivalent to the following configuration:

componentsGroupMapping {
  base: "Basic Components"
  sources: "Sources"
  sinks: "Sinks"
  services: "Services"
  enrichers: "Enrichers"
  custom: "Custom"
  optionalEndingCustom: "Optional Ending Custom"
  statefulTransformers: "Stateful Transformers"
  miscellaneous: "Miscellaneous"
  fragments: "Fragments"
  fragmentDefinition: "Fragment Definition"
  deprecated: "Deprecated"
}

You can override this default grouping of components in toolbox panels with the componentsGroupMapping setting. Your entries take precedence over the defaults, and only the groups you mention are affected - the remaining ones keep their default names. Mapping a group to null hides its components. Group names which are not listed above - for example groups introduced with the componentGroup property - are displayed as they are, unless you map them here.

Keys and values are not written the same way:

  • a key is the group a component is assigned to - the raw name, as in the left column above (base, enrichers, ...),
  • a value is always the final name of the toolbox panel, exactly as it is displayed.

Mapping is resolved in one step and is not applied again to its own result, so to merge one group into another you must name the target by its final name. To put the custom components into the Basic Components panel, write custom: "Basic Components" - custom: "base" would instead create a second panel literally named base, next to Basic Components.

componentsGroupMapping {
  enrichers: "Data enrichment"
  custom: "Basic Components"
  deprecated: null
}

If you use this configuration, you must place it repeatedly in all configured processing types.

Scenario properties​

It's possible to add additional properties for scenario. They can be used for allowing more detailed scenario information (e.g. pass information about marketing campaign target etc.), they can also be used in various Nussknacker extensions:

Example (see dev application config for more examples):

scenarioPropertiesConfig {
  campaignType: {
    editor: { type: "SpelTemplateParameterEditor" }
    validators: [ { type: "MandatoryExpressionValidator" } ]
    label: "Campaign type"
    defaultValue: "Generic campaign"
  }
  ...
}

Configuration is similar to component configuration

The docs button which is displayed in the scenario properties modal window is by default disabled for properties. One can enable it by adding

scenarioPropertiesDocsUrl: "http://custom-configurable-link"

in the config.

Multitenancy support​

Nussknacker supports multitenancy, allowing multiple Nussknacker designer instances to operate on shared infrastructure components, such as Kafka, Flink, or InfluxDB. This is achieved by using configured namespaces to isolate resources:

  • Kafka Topics: Prefixed with a namespace
  • Kafka Consumer Groups: Prefixed with a namespace
  • Flink Jobs: Prefixed with a namespace
  • InfluxDB Metrics: Tagged with an additional namespace tag

A namespace configuration can be defined as follows:

modelConfig: {
  ...
  namespace: customer1
  namespaceSeparator: "_"
}

Notes:

  • The namespaceSeparator is optional and defaults to _ if not specified.
  • This configuration applies a uniform namespace to all resources.