Nussknacker is production ready - it's been used in major Polish telcom since beginning of 2017. However, it's still under active development, so some parts of the API may change in the future.
For detailed instructions how to migrate to the newest version please see migration guide.
1.1.2 (not released yet)
- #2920 Close periodic engine actors. Reverse processing type reload - close and then reload.
- #2660 Fix for handling errors after split in async mode
- #2744 Ugly resource waste fixed in component drag preview
- #2754 Fix error with pasting node,
- #2807 Fix default values for GenericNodeTransformation
- #2176 Allow to enrich periodic scenario config on initial schedule and each deployment.
- #2179 Permission allowing for editing scenario on FE, but not saving etc.
- #2150 Better handling of multiple schedules in batch periodic engine - fixed running one time scenarios and improved current scenario status reporting.
- #2208 Upgrade libraries: cats 2.6.x, cats-effect 2.5.x, circe 0.14.x
- #1422 Remove
WithExplicitMethod, added helpers, small refactor
- #2278 SQL Variable is removed
- #2280 Default values for parameters can be setup programmatically now - thanks to
- #2293 Enhancement: change
- #2169 Add Apache Ignite support to SQL Component by implementing a custom DB metadata provider that extends the standard JDBC Driver with missing features.
- #2301 #2366#2409 #2477 Simplification of component API:
GenericNodeTransformation.fallbackFinalResultintroduced for not handle step, with default graceful strategy
ParameterValidatorproperly. Invalid value is handled as
GenericNodeTransformation.implementationis not invoked in this case
FinalResults.forValidationutility method added to easily handle situation when you need to make some validation on context of variables (e.g. add variable checking if it already exists)
- #2245 Periodic process scheduler retries failed scenario deployments based on PeriodicBatchConfig. Breaking change in PeriodicProcessListener FailedEvent. Failed event is split into FailedOnDeployEvent and FailedOnRunEvent. Please note that this mechanism only retries when failure on deployment occurs - failure recovery of running scenario should be handled by restart strategy
- #2304 Upgrade to Flink 1.14
FlinkLazyParameterFunctionHelperhas additional methods to handle exceptions during evaluation gracefully
- #2300 Enhancement: refactor and improvements at components group
- #2347 Support for implicit type conversions between
Stringand various value classes (
- #2346 Remove
- #2331 #2496 Refactor
nussknacker-avro-flink-utilmodule. Move non-flink specific classes to new
- #2348 #2459 #2486#2490 #2496
nussknacker-kafka-flink-utilmodule. Move non-flink specific classes to
- #2344 Redesign of
- #2305 Enhancement: change
- #2374 Auto-loaded
- #2337 Extract base engine from standalone
- Common functionality of base engine (i.e. microservice based, without Flink) is extracted to
- It's possible to use generic effect type instead of
- Possibility to accumulate errors
- New API for custom components (transformers and sinks)
- Common functionality of base engine (i.e. microservice based, without Flink) is extracted to
- #2349 Removed module
FlinkQueryableClientwas moved to
- #2361 Removed
LoggedUserreplaced with dedicated class in
- #2367, #2406 Simple kafka-based streaming scenario interpreter. Stateless, with basic kafka sinks and sources. This is MVP, not intended for direct usage, more work with sources, sinks and invoking will come in next PRs
- #2377 Remove
SourceFactory. It was used mainly for test sources.
- #2534 Remove generic parameter from
SourceFactory. It was used mainly to determine
- #2397 Common
MetricsProvider, cleaning unnecessary dependencies on Flink
- #2486 Aggregates now producing context id in similar format as sources - will be visible in "Test case" during usage of tests mechanism
- #2465 aggregate-sliding emitWhenEventLeft parameter changed default value from true to false
- #2474 Interpreter return type changed from
F[Either[List[InterpretationResult], EspExceptionInfo[_ <: Throwable]]]to
F[List[Either[InterpretationResult, EspExceptionInfo[_ <: Throwable]]]]. Hence, e.g. multiple branches in Graph can be evaluated, both positively and negatively at the same time.
- #2540 It's possible to use different Effects than
Futurein request-response (standalone) runtime.
InvocationMetricsare no longer automatically computed, as they are
Futuredependent - see
StandaloneRequestHandlerhow to enable them.
ServiceLoadermechanism to load additional encoders.
- #1439 Upgrade to Flink 1.13
- #1993 Demo was moved to https://github.com/TouK/nussknacker-quickstart. Some additional refactors done: logback configuration enhancements, simpler run.sh script, removed docker defaults from default configs.
- #2105 #2112 Better handling Flink's job deploying - we report job initialization as a "DURING_DEPLOY" instead of "RUNNING" now, and we are checking available slots on Flink before deploy
- #2152 Possibility to create
SchedulePropertyExtractorusing deployment manager's configuration.
- #2133 SQL Variable is hidden in generic model
- #2101 Global permissions can be arbitrary string, can configure all top tabs (by default
- #2103 Counts work correctly with different timezones,
- #2104 SQL component can retrieve table names for completion
- #2028 Limit (row and bytes) for generating and using test data
- Various improvements in security/OAuth components
- #2046 Additional functions in generic model
- Security improvements:
- #2095 Remove business view
- #2110 Remove node grouping
- #2098 Correct timestamps for tests of Kafka sources
- #2108 Enhanced class extraction settings, fewer unnecessary methods
- #2191 KafkaAvroSink performance fix
- UI enhancements:
- #1706 New window manager, consistent behaviour, many improvements, e.g. modals can be expanded to full screen, fix display of fragments in FF
- #2184, #2101 Fix undo breaking UI in certain circumstances
- #2181, #1975 Remove spurious 'unsaved changes' after opening aggregation nodes
- #2202 Correct hashes of FE assets
- #2097, #2178 Pasting nodes in correct places
- #2003 Counts dialog fixes: timezone handling, datepicker allows editing from keyboard
- #2111 Correct graph display after opening fragment
- #2087 Pan and zoom animation
- #2081 Fix switch behaviour after changing condition
- #2071 Fix pasting cell on multiple edges
- #1978 Removed unclear node details panel
- More precise TypeInformation generation
- #1361 Lazy vars removal
- #1363 Open/close only services that are actually used in process
- #1367 Custom actions - first, experimental version
- Migration of CI to github actions
- #1382 First E2E FE tests
- #1373 Ability to load custom model config programmatically
- #1406 Eager services - ability to create service object using static parameters
- #962 New ways of querying InfluxDB for counts, integration tests, no default database name in code
- #1428 Kafka SchemaRegistry source/sink can use JSON payloads. In this PR we assume one schema registry contains either json or avro payloads but not both.
- #1445 Small refactor of RecordFormatter, correct handling different formatting in kafka-json in test data generation
- #1433 Pass DeploymentData to process, including deploymentId and possible additional info
PeriodicProcessListenerallows custom handling of
ProcessManagerAPI allows to return ExternalDeploymentId immediately from deploy
- #1405 'KafkaAvroSinkFactoryWithEditor' for more user-friendly Avro message definition.
- #1514 Expose DeploymentData in Flink UI via
FlinkSourceAPI allows to create stream of
Context(FlinkSource API and test support API refactoring).
- #1497 Initial support for multiple (named) schedules in
- #1499 ClassTag is provided in params in avro key-value deserialization schema factory:
- #1533 Fix: Update process with same json
- #1546 Unions (e.g after split) are possible in standalone mode. Also, it's possible to define transformers which operate on all results (e.g. for sorting recommendations)
- #1547 Publish first version of BOM including dependencyOverrides
ComponentProviderAPI enables adding new extensions without changing e.g.
- #1471 Initial version of session window aggregate added (API may change in the future).
- #1631 Ability to use multiple config files with
KafkaSourceFactoryis replaced with source that provides additional #inputMeta variable with event's metadata.
- #1663 Flink restart strategies and exception consumers can now be configured.
- #1728 Replace schemaRegistryClient and recordFormatter in
SchemaRegistryProviderwith their factories.
KafkaAvroSourceFactoryprovides additional #inputMeta variable with event's metadata.
TypingResultAwareTypeInformationDetectioncan be used to serialize aggregates more efficiently
- #1772 Fix for Spel validation when we try use not existing method reference
- #1741 KafkaExceptionConsumer can be configured to send errors to Kafka
- #1809 Performance optimization for aggregates: do not update state if added element is neutral for current state
- #1886 Performance optimization for aggregates: do not save context in state. Added
#AGGutility for easier switching from simple aggregating functions like
'Sum'to more complex
- #1820 Added missing support for some logical types (LocalDate, LocalTime, UUID) in json encoding
- #1799 ConfluentAvroToJsonFormatter produces and reads test data in valid json format with full kafka metadata and schema ids.
- #1839 Set up
explicitUidInStatefulOperatorsmodel's flag to
- #1357 Add run mode to nodes to be able to determine if we are inside e.g. test process run.
Run mode is can be declared as a dependency in generic node transformations. Nodes created via
RunModeas an implicit parameter.
RunModeis also available in
- Various naming changes:
- Performance fixes:
- #1321 Exception handler accessible via custom node context, avro record encoding errors reported by exception handler
- #1298 Feature flag
avroKryoGenericRecordSchemaIdSerializationfor avro kryo serialization optimization (default = false)
- #1315 Spring bumped 5.1.4 -> 5.1.19
- #1312 Ficus bumped 1.4.1 -> 1.4.7
- #1288 Namespaces can be configured for ObjectNaming
- #1261 Fix: Access to
Property or field cannot be found on object
- #1244 Ability to define
- #1165 Typed global variables
- #1128 Union-memo transformer
- #1054 Tabbed dark process list
- Configuration improvements (library upgrade, conventions): #1151, #1166
- #873, #1044 Flink upgrade (to 1.11)
- More graceful handling of Flink compatibility issues (in particular, `
FlinkCompatibilityProvidertrait introduced, also ProcessManager implementations are separated from UI to allow easier changes in deployments): #1150, #1218
- #1183 New back to process button on metrics
- #1188 Fix env label and provide nussknacker logo
- #249 Inferred expression type in node modal
- #1255 Moved displaying
- #1257 Improvements: Flink test util package
- #1287 OAuth2: add accessTokenRequestContentType parameter
- #1290 Own kryo serializers can be provided through SPI
- #1303 TypedObjectTypingResult can have additional info (e.g. Schema for GenericRecord)
- #1175 Fix for: BestEffortAvroEncoder haven't produced record with logical types for missing field with default values
- #1173 Fix for: Avro source wasn't be able to read record with schema with invalid defaults
- #1127 Fix too small count values
- #1133 Improvements: More flexible TestReporter instancies implementation
- #1131 Fix: Disable "deploy" & "metrics" buttons for subprocess
- #1148 Fix FE regexp for match node id
- #1099 New outer-join node
- #1024 Added default async interpretation value configured by
asyncExecutionConfig.defaultUseAsyncInterpretation(false if missing).
- #879 Metrics can now use Flink variables for better reporting, it's recommended to use InfluxDB native protocol instead of legacy Graphite protocol to send metrics to InfluxDB.
- #940 More detailed node errors
- #949 JVM options can be configured via JDK_JAVA_OPTIONS env variable (in docker and standalone distribution)
- #954 Correct handling of types in empty inline lists
- #944 System cache mechanism
- #704 Preloaded creator panel node icons
- #943 Literal min / max validators
- #976 Fixed save button & groups expand for businessView
- #973 Textarea editor
- #987 Optimized graph rendering time, fixed minor bugs (expand group icon, view center & fit after layout).
- Introduction to KafkaAvro API: #871, #881, #903, #981, #989, #998, #1007, #1014, #1041,
- Performance improvements in interpreter: #1008, #1013. The second one also removes Future from expression evaluation
- Dynamic parameters: filter validation, GenericNodeTransformation introduction (for CustomNodes, Sources, Sinks) - also handling dynamic parameters on UI: #978, #996, #1001, #1011
- #988 Json editor
- #1066 Duration and period editors fixes
- #1126 New nodes: periodic source, delay and dead-end
- #965 Added new, 'aggregate-tumbling' node.
- #957 Custom node
aggregatehas now additional aggregation function
Sum. Also was changed parameter from
windowLengthwith human friendly duration input.
- Branch parameters now can be eager (computed during process compilation)
- More restrictive type checking in SpEL - mainly added verification of types of method's paramaters
- Added support for Kafka consumer group strategies - setted up by
- Bugfixes for joins
- Added support for explicitly setting uids in operators - turned on by
explicitUidInStatefulOperatorsmodel's flag. By default setted up to false.
- Old way of configuring Flink and model (via
processConfig) is removed.
processTypesconfiguration should be used from now on.
- Change of additional properties configuration
0.0.12 (26 Oct 2019)
- Cross builds with Scala 2.11 and 2.12
- First version of join nodes
- OAuth2 authentication capabilities
- Migration of Argonaut to Circe
- Preliminary version of dictionaries in expressions
- Major upgrade of frontend libraries (React, Redux, etc)
- Various usability improvements
0.0.11 (1 Apr 2019)
0.0.10 (13 Nov 2018)
0.0.9 (13 Jul 2018)
0.0.8 (7 May 2018)
- expressions code syntax highlighting
- source/sink params as expressions
- multiline expression suggestions
- method signature and documentation in code suggestions
- inject new node after dragging on edge
- Query services tab in UI
- subprocess disabling
- display http request-response for query service tab
- flink kafka 0.11 connector
- dynamic source return type
- SQL can be used as expression language
- Processes page rendering optimized
- suggestions for projections/selections in spel
- upgrade to flink 1.4.2
- upgrade to scala 2.11.12
- Make sinks disableable
0.0.7 (22 Dec 2017)
- global imports in expressions
- deployment standalone on multiple nodes
- typed SpEL expressions - first iteration
- can post process standalone results
- support for java services
- handling get requests in standalone mode
- metric fixes for standalone
- compare with other env
- split in request/response mode by expression
- ProcessConfigCreator Java API support added
- extendable authentication
- comparing environments - first part, can compare processes
- subprocess versions
- process migrations + some refactoring
- async execution with toggle
- better exception for errors in service invocations
- nussknacker java api
- spring version bump because of SPR-9194
0.0.6 (9 Aug 2017)
First open source version :)