1.13.7-beta.1

What's Changed

• fix(sql): backport CVE-2024-43406 by @Crispy-fried-chicken in #3130
• feat: support kafka writer batch conf by @Yisaer in #3495
• chore(build): remove android build by @ngjaying in #3496
• chore(build): update downland artifact by @ngjaying in #3497

New Contributors

• @Crispy-fried-chicken made their first contribution in #3130

Full Changelog: 1.13.6...1.13.7-beta.1

v2.1.0-alpha.2

What's Changed

• fix: fix replace duration by @Yisaer in #3400
• fix(log): modify nng sock log by @ngjaying in #3403
• feat: support align incremental window by @Yisaer in #3399
• chore: merge exp branch by @ngjaying in #3402
• refactor(neuron): adapt to multi tags by @ngjaying in #3405
• fix: async import data return response by @Yisaer in #3408
• fix: replace props when get by @Yisaer in #3409
• fix: fix flaky TestFileDirSource by @Yisaer in #3412
• fix: fix incrmental sliding window delay function by @Yisaer in #3411
• fix(mqtt): conn subs npe by @ngjaying in #3415
• feat: add source span tag by @Yisaer in #3413
• fix: fix incremental sliding delay window by @Yisaer in #3417
• feat: support incremental window state by @Yisaer in #3406
• feat(converter): support binary encode by @ngjaying in #3420
• fix(window): sliding delay time by @ngjaying in #3424
• feat: support incremental event sliding window by @Yisaer in #3421
• feat: remove promethues tag by @Yisaer in #3427
• fix(io): file source ignore endLine buffer by @ngjaying in #3431
• feat: support incremental event hopping window by @Yisaer in #3429
• fix(state): untriggerd rule always cancel by @ngjaying in #3436
• fix(sql): remove hot path log by @Yisaer in #3438
• feat: revise metrics code by @Yisaer in #3430
• feat: support event tumbling/count window by @Yisaer in #3434
• build(deps): Bump golang.org/x/crypto from 0.27.0 to 0.31.0 by @dependabot in #3439
• feat: support incremental window state by @Yisaer in #3440
• feat: revise grafana panel by @Yisaer in #3452
• docs: import dashboard in grafana by @Yisaer in #3453
• fix: fix sync cache context by @Yisaer in #3451
• fix: converter lose precision by @Yisaer in #3445
• feat(io): file source support inotify subscribe by @ngjaying in #3448
• fix: fix xml converter panic by @Yisaer in #3455
• feat: support event inc window state by @Yisaer in #3446
• feat: support incremental avg by @Yisaer in #3457
• docs: add incremental doc by @Yisaer in #3462
• feat: push down alias decode by @Yisaer in #3460
• chore: merge exp features and fixes by @ngjaying in #3467
• fix(win): sliding delay lost previous input by @ngjaying in #3471
• feat: support inc agg function by @Yisaer in #3465
• feat: upgrade base image to 1.23.4 by @Yisaer in #3479
• feat: support config portable timeout by @Yisaer in #3464
• feat: add alias reference calculate swtich by @Yisaer in #3477
• fix: downgrade sqlite to restore arm7 support by @Yisaer in #3481
• feat(io): merge dirwatch into file source by @ngjaying in #3480
• feat: upgrade go 1.23.4 by @Yisaer in #3484

Full Changelog: v2.1.0-alpha.1...v2.1.0-alpha.2

v2.0.6

What's Changed

• fix(io): mqtt subscription concurrent map by @Yisaer in #3475
• fix: downgrade sqlite to restore arm7 support by @Yisaer in #3482

Full Changelog: v2.0.5...v2.0.6

v2.0.5

What's Changed

• fix(state): untriggerd rule always cancel by @ngjaying in #3435
• chore: merge bugfixes to v2.0.5 by @ngjaying in #3469
• chore: update package for security by @ngjaying in #3470
• chore(window): revert warn log to debug by @ngjaying in #3472

Full Changelog: v2.0.4...v2.0.5

v1.14.7

What's Changed

• feat: support wait sink node close by @Yisaer in #3373
• chore: bump go_modules/golang.org/x/crypto-0.31.0 by @OlgasAcc in #3447

New Contributors

• @OlgasAcc made their first contribution in #3447

Full Changelog: v1.14.6...v1.14.7

v2.0.2

What's Changed

• fix: replace props when get (#3409)
• fix: async import data return response (#3408)
• refactor(neuron): adapt to multi tags
• fix(op): merger use merge time as et
• fix(op): send out batch when EOF
• fix(op): avoid rate limit panic

Full Changelog: v2.0.1...v2.0.2

v2.1.0-alpha.1

Features

• feat(enc): support aes gcm by @ngjaying in #3344
• feat(rule): support send out nil fields by @ngjaying in #3342
• Incremental window
  • feat: support inc agg window by @Yisaer in #3322
  • feat: support incremental window in having by @Yisaer in #3358
  • feat: support tumbling/sliding inc window by @Yisaer in #3361
  • feat: support inc window filter by @Yisaer in #3379
  • feat: support incremental hopping window by @Yisaer in #3386
• feat: support xml converter by @Yisaer in #3341
• feat(func): add reverse function by @Dream95 in #3357
• feat: support kafka sink compression by @Yisaer in #3374
• feat: support auth request headers by @Yisaer in #3376
• feat: validate confkey by @Yisaer in #3378
• feat: rewrite duration in conf by @Yisaer in #3387
• Support MQTT 5
  • feat(io): support mqtt5 by @ngjaying in #3384
  • feat(mqtt): v5 sink supports properties by @ngjaying in #3390
  • feat(trace): support mqtt 5 trace by @ngjaying in #3394
• feat: support merge tokens into body by @Yisaer in #3392
• feat: support fileDir notify source by @Yisaer in #3380
• feat: support sql lookup template sql by @Yisaer in #3398

What's Changed

• docs(rule): doc of triggered prop by @ngjaying in #3338
• chore: sendError defaults to false by @ngjaying in #3339
• ci: fix doc version by @ngjaying in #3340
• fix(func): object_construct always issue keys by @ngjaying in #3345
• fix: remove tryrun in ruletest by @Yisaer in #3348
• chore(test): change video by @ngjaying in #3354
• build: fix make lint error by @Dream95 in #3363
• fix: sql source support null value by @Yisaer in #3365
• fix(conn): conn status nil exception by @ngjaying in #3372
• fix(planner): clean subtopo when creation fail by @Yisaer in #3383
• fix(kafka): fix dynamic prop problem by @ngjaying in #3391
• fix(neuron): avoid flooding logs when disconnect by @ngjaying in #3393
• docs: add kafka sink doc by @Yisaer in #3396

Full Changelog: v2.0.0...v2.1.0-alpha.1

v2.0.1

What's Changed

• docs(rule): doc of triggered prop by @ngjaying in #3338
• chore: sendError defaults to false by @ngjaying in #3339
• ci: fix doc version by @ngjaying in #3340
• fix(func): object_construct always issue keys by @ngjaying in #3345
• fix: remove tryrun in ruletest by @Yisaer in #3348
• chore(test): change video by @ngjaying in #3354
• build: fix make lint error by @Dream95 in #3363
• fix: sql source support null value by @Yisaer in #3365
• fix(conn): conn status nil exception by @ngjaying in #3372
• feat: support kafka sink compression by @Yisaer in #3374
• feat: support auth request headers by @Yisaer in #3376
• feat: validate confkey by @Yisaer in #3378
• fix(planner): clean subtopo when creation fail by @Yisaer in #3383
• feat: rewrite duration in conf by @Yisaer in #3387
• fix(kafka): fix dynamic prop problem by @ngjaying in #3391
• feat: support merge tokens into body by @Yisaer in #3392
• fix(neuron): avoid flooding logs when disconnect by @ngjaying in #3393

Full Changelog: v2.0.0...v2.0.1

v1.14.6

What's Changed

• fix: fix sql converter by @Yisaer in #3368
• fix: larger scanner buffer by @Yisaer in #3369

Full Changelog: v1.14.5...v1.14.6

v2.0.0

IO(Source/Sink)

• Connection management https://ekuiper.org/docs/en/latest/guide/connections/overview.html

  • Connection pool framework (gradually adapt existing connections)
    • Connection become a physical instance instead of a rule logical configuration, it will run from the start and refered by multiple rules
    • Standalone connection status
    • Auto reconnect
  • Adapt mqtt, sql, nng(neuron), edgex, websocket, httppush
  • APIs to manage connection instances
  • Connection status metrics

• Source/sink operators split source split/sink split

  • Finer-grained runtime metrics can be supported to understand the status and latency of each sub-task.
  • Parallel computation of sub-tasks can be implemented, improving the overall efficiency of rule execution.

• Source down sampling support doc

• Formats

  • Delimited format support header
  • Add urlencoded format (usually use in HTTP form request, extract from REST sink)

• Multi-level decode (format and payloadFormat)

• File sink supports enctryption (AES now)

• REST sink supports full format like delimited(csv), also add validation for mapping of format and bodyType

SQL/Streaming runtime

• General ONNX function plugin, run ONNX model inference without coding doc
• Support stream EOF so that we can run a rule in batch mode
  • File/simulator source stops after read all and no loop/interval is set
  • MQTT stream allows to stop by sending EOF message
• Rule gracefully exit: wait until all operators close
• New function: object_size
• Support window function doc
• Datetime format, support multiple S in format layout

Operation

• Tracing support doc
  • REST API tracing
  • Query rule trace API
  • Tracing data from/to Neuron
  • Local trace
  • Tracing collector
• Rule state cleanup
  • View rule metrics after close
  • Avoid malform state
• New rule metrics: all metrics now
  • connection related metrics for source/sink
• CPU usage metric
• API updates
  • Import
    • Asnc API
    • Supports YAML content
  • Meta list return plugin type info
  • List streams API show format and type
  • Drop dependency validation
    • stream
    • plugin
• Store conf in KV by default
• Conf: use time.Duration for intervals

Extension

• Native plugin develop
  • Update extention spec, users need to rewrite plugins
  • Do not need to depend on eKuiper project any more
• Portable plugin
  • Support sink ack so that it can leverage sink cache doc
  • Start as long running instance
  • Plugin status API, reference count
  • Remove recv size limit (won’t fail when communication payload is bigger than 1MB)
• Support REST external service retry

Test & Build

• Upgrade to go 1.23
• Build: add docker alpine-python image
• UT coverage 71%
• Security
  • Fix integer truncate problem
  • SQL prepare to avoid injection
  • Escape html response

What's Changed

• chore: remove repetitive words by @InventiveCoder in #2718
• build: upgrade base image go version by @ngjaying in #2720
• feat: introduce rule metrics by @Yisaer in #2716
• build: upgrade go and dependant mods by @ngjaying in #2719
• feat: refact sql indexValue into muti indexValue by @Yisaer in #2717
• refactor(xsql): unify row and collection by @ngjaying in #2700
• fix: fix doc error by @Yisaer in #2724
• feat(module): modularize io by @ngjaying in #2721
• feat: add multi field index api by @Yisaer in #2729
• feat(module): modularize converter by @ngjaying in #2728
• docs: fix some errors by @L-607 in #2731
• feat: let IntegerLiteral Value as int64 by @Yisaer in #2727
• fix: fix describe rule exposes password by @Yisaer in #2732
• docs: add doc for multiple sql index field by @Yisaer in #2733
• docs: fix websocket doc path by @Yisaer in #2735
• feat(modularize): modularize function by @ngjaying in #2737
• chore(ci): fix tag list error by @Swilder-M in #2738
• fix: replace password when create rule by @Yisaer in #2736
• feat: support nested array schema by @Yisaer in #2734
• fix(planner): avoid create subtopo in error by @ngjaying in #2740
• fix: check rule exists before create by @Yisaer in #2741
• fix: redisPubSub ping by @retoool in #2739
• fix: doc problem for tdengine sink by @carlclone in #2743
• feat: add security to edgex via openziti by @dovholuknf in #2603
• refactor: split sink batch by @ngjaying in #2745
• fix: fix load stream when meet error by @Yisaer in #2751
• ci: delete v prefix in git tag by @Rory-Z in #2752
• fix: ignore nil value in array convert by @carlclone in #2748
• fix: fix template sql index value update strategy by @Yisaer in #2754
• feat: support modify sqlSource field index value by @Yisaer in #2730
• feat: support rule stop by error metrics by @Yisaer in #2755
• feat: package v2 by @ngjaying in #2766
• refactor: move api to standalone go mod by @ngjaying in #2770
• chore(temp): remove extensions by @ngjaying in #2777
• feat(v2): mvp features by @ngjaying in #2780
• feat: new source/sink v2 API by @ngjaying in #2783
• refactor(contract): keep interfaces only by @ngjaying in #2790
• chore: refer to correct contract mod by @ngjaying in #2792
• fix(test): source/sink test update by @ngjaying in #2797
• feat(api): add error ingest for source subscribe by @ngjaying in #2800
• refactor(sink): split compression op by @ngjaying in #2802
• feat(sink): split encryptor by @ngjaying in #2807
• feat(op): add dedup trigger op by @ngjaying in #2801
• refactor(converter): add context to converter API by @ngjaying in #2812
• feat(sink): add back file sink by @ngjaying in #2811
• feat: separate schema layer from decoder by @Yisaer in #2815
• refactor(sink): mqtt sink use connection pool by @ngjaying in #2814
• feat: remove useless flag in conveter by @Yisaer in #2823
• feat(sink): support file sink rolling hook by @ngjaying in #2819
• refactor: provide schema when instantiating by @ngjaying in #2821
• chore: update contract version by @ngjaying in #2826
• feat(function): object pick enhancement (#232) by @ngjaying in #2833
• refactor(model): extract raw tuple by @ngjaying in #2835
• feat(sink): add sink dynamic properties support by @ngjaying in #2837
• refactor(op): decode op schema more logs by @ngjaying in #2840
• chore: update codecov action by @ngjaying in #2843
• fix: detach for non shared connection by @ngjaying in #2845
• refactor(sink): split cache op by @ngjaying in #2852
• fix: revise interval config by @Yisaer in #2839
• feat(cache): resend alter queue support by @ngjaying in #2855
• fix: let config compatible by @Yisaer in #2861
• refactor(node): use time.Duration and time.Time by @ngjaying in #2864
• fix: support wait operators close by @Yisaer in #2862
• feat(io): add file source in lines mode by @ngjaying ...