v1.25.5

This is a patch release.

What's Changed

1. Fix crash when no SEPARATOR is specified in the FT.CREATE command (#4205)
2. Fix crash when defragmenting StringSet (#4283)
3. Fix crash caused by destructor in RaxTreeMap (#4228 )

Full Changelog: v1.25.4...v1.25.5

v1.25.4

This is a patch release.

What's Changed

1. fix: rdb load fix loading plain lists 4959bef
2. fix conflict on cherry-pick rdb load fix 976586c

full change log: v1.25.3...v1.25.4

v1.25.3

This is a patch release.

What's Changed

1. fix: stop using openssl for container healthchecks (#4181)
2. fix: deduplicate mget response (#4184)
3. fix(tools): Prevent dragonfly.logrotate to stop logrotate service (#4176)
4. fix: remove DenseSet::IteratorBase::TraverseApply (#4170)
5. chore: Hide replicas from CLUSTER subcmds in managed mode (#4174)

Full Changelog: v1.25.2...v1.25.3

v1.25.2

This is a patch release.

1. fix: Huge entries fail to load outside RDB / replication (#4154)
   fixes #4143
2. chore: optimize info command (#4137)
   decrease latency for "info server" and "info replication" commands
3. fix: Use MOVED error type for moved replies (#4125)
   Use type MOVED for moved replies, making them all the same under # Errorstats
   Fixes #4118
4. fix: a potential crash during client list (#4141)

v1.25.1

This is a patch release.

fix: regression in squashing code when determining eval commands (#4116)

The regression was caused by #3947 and it causes crashes in bullmq.
It has not been found till now because python client sends commands in uppercase.
Fixes #4113

Signed-off-by: Roman Gershman [email protected]

v1.25.0

Dragonfly v1.25.0

Some prominent changes include:

• FT.SEARCH fixes: #3956 #3955 #4043
• Prefix search - supporting queries like "@color:{gre* | orange | yellow*}: #3972
• GCS save/load support #4006
• Few Improvements around replication/snapshotting - it should be more efficient around large entries #4041
• zrevrank/zrank support withscore option #4001

What's Changed

• chore: ignore applying the same cluster config twice by @BorysTheDev in #3932
• test: add test to reproduce a lot of memory consumtion during migration by @BorysTheDev in #3939
• chore(refactoring): header clean ups by @romange in #3943
• chore: get rid of ToUpper call and use AsciiStrToUpper by @romange in #3944
• chore: remove ToUpper calls in main_service by @romange in #3947
• chore: get rid of ToUpper/ToLower mutations on arguments by @romange in #3950
• chore(facade): Fix bad new IO glue by @dranikpg in #3940
• feat: track request sizes histograms by @romange in #3951
• chore: pass SinkReplyBuilder and Transaction explicitly. Part1 by @romange in #3946
• chore: disable test_cluster_memory_consumption_migration by @kostasrim in #3948
• fix(search_family): Temporary remove the error when a field name does not have the '@' sign at the beginning in the FT.AGGREGATE command by @BagritsevichStepan in #3956
• chore: tune test_rss_used_mem_gap by @kostasrim in #3958
• test: add test to test big collections or collections with big values by @BorysTheDev in #3959
• chore: increase load in test_noreply_pipeline by @kostasrim in #3960
• chore(fakeredis): Enable JSON tests in the Fakeredis tests by @BagritsevichStepan in #3773
• chore: add defrag logic for zsets by @kostasrim in #3836
• chore: pass SinkReplyBuilder and Transaction explicitly. Part2 by @romange in #3954
• chore: pass SinkReplyBuilder and Transaction explicitly. Part3 by @romange in #3966
• fix: crash if we OOM during migration process by @BorysTheDev in #3968
• fix(cluster): fix backoff on unknown migration by @andydunstall in #3965
• chore: pass SinkReplyBuilder and Transaction explicitly. Part4 by @romange in #3967
• chore: get rid of MutableSlice by @romange in #3952
• chore: reduce dependency of debug/memory commands on ConnectionContext by @romange in #3977
• fix: properly set object with DenseLinkKey by @romange in #3980
• fix(memory-leak): add missing destructor to RaxTreeMap by @kostasrim in #3975
• fix: dangling string in DebugCmd::Reload by @kostasrim in #3981
• chore: pass SinkReplyBuilder and Transaction explicitly. Part5 by @romange in #3976
• chore: pass SinkReplyBuilder and Transaction explicitly. Part6 by @romange in #3987
• fix(build): Reduced checks permission in build to read #3821 by @irufus in #3916
• fix: properly set ttl bit during object replacement by @romange in #3991
• fix: replace string_view with string in RaxTreeMap::FindIterator by @kostasrim in #3982
• fix: prevent starting migrations from replica by @BorysTheDev in #3985
• feat(search): Prefix search for tags by @dranikpg in #3972
• chore: pass SinkReplyBuilder and Transaction explicitly. Part8 by @romange in #3992
• chore: pass SinkReplyBuilder and Transaction explicitly. Part7 by @romange in #3988
• chore: pass SinkReplyBuilder and Transaction explicitly. Part9 by @romange in #3996
• fix: keep the ttl bit when dense_set grows with expiring items by @romange in #3995
• refactor(workflow): changes default permissions to read for few workflows increasing security by @BLANKatGITHUB in #3905
• fix(search_family): Fix indexes loading in the FT.SEARCH and Ft.AGGREGATE commands by @BagritsevichStepan in #3955
• feat: introduce metrics/logs of when pipelining is being throttled by @romange in #4000
• chore: clean up conn_use_incoming_cpu heuristic by @romange in #3978
• chore: refactor snapshot expanding logic by @romange in #4003
• chore: introduce GcsSnapshotStorage by @romange in #4004
• fix: flaky test_failover test by @romange in #4007
• chore: pass SinkReplyBuilder and Transaction explicitly. Part10 by @romange in #3998
• fix: build break in search_family by @romange in #4008
• feat(zset_family): support WITHSCORE in zrevrank/zrank commands (#3921) by @Diskein in #4001
• fix: separate Heartbeat and ShardHandler to fibers by @kostasrim in #3936
• chore: improve verbosity logging during shutdown by @romange in #4010
• chore: skip AggregateWithLoadOptionHard on sanitizers (false positive) by @kostasrim in #4013
• chore: pass RedisReplyBuilder explicitly from dragonfly connection by @romange in #4009
• chore: print info stats if test_noreply_pipeline fails by @romange in #4016
• chore: flush journal to channel for non auto journal commands by @kostasrim in #3945
• chore: support load/save from GCS by @romange in #4006
• chore: pull helio by @romange in #4036
• chore: decrease logs severity for OOM from error to warning by @BorysTheDev in #4031
• feat(pre-commit): adds senstive data check like api keys by @BLANKatGITHUB in #4030
• test: update test_big_containers by @BorysTheDev in #4025
• fix: reduce memory consumption during migration by @BorysTheDev in #4017
• chore: add tx queue head debug info in AnalyzeTxQueue by @kostasrim in #4026
• chore: refactor part of s3 logic for loading a snapshot. by @romange in #4044
• chore: add retry to test_noreply_pipeline test by @romange in #4045
• chore(DenseSet): defrag all links in a chain by @kostasrim in #4019
• chore: add ForceUpdate to BPTree by @kostasrim in #3993
• fix(search_family): Fix crash in FT.PROFILE command for invalid queries by @BagritsevichStepan in #4043
• feat(dependabot): Add dependabot to check for vulnerabilies and updat… by @BLANKatGITHUB in #4035
• chore: pass reply_builder explicitly to pubsub module by @romange in #4021
• chore(deps): bump numpy from 1.24.1 to 2.1.3 in /tools by @dependabot in #4063
• chore(deps): bump github.com/redis/go-redis/v9 from 9.5.1 to 9.7.0 in /tools/replay by @dependabot in #4062
• chore(deps): bump tomli from 2.0.1 to 2.0.2 in /tests/dragonfly by @dependabot in #4059
• chore(deps): bump github.com/gruntwork-io/terratest from 0.41.11 to 0.47.2 in /contrib/charts/dragonfly by @dependabot in #4058
• chore(deps): bump redis-om from 0.2.2 to 0.3.3 in /tests/dragonfly by @dependabot in #4060
• chore(deps): bump mikepenz/action-junit-report from 4 to 5 by @dependabot in #4052
• chore(deps): bump mozilla-actions/sccache-action from 0.0.4 to 0.0.6 by @dependabot in #4051
• chore(deps): bump ossf/scorecard-action from 2.3.1 to 2.4.0 by @dependabot in #4053
• chore(deps): bump github/codeql-action from 3.24.9 t...

v1.24.0

Dragonfly v1.24.0

It's a maintenance release with mostly performance and bug fixes for edge cases.

Some prominent changes include:

1. feat: add HEXPIRE and FIELDEXPIRE by @NegatioN #3780
2. fix BITPOS command responses by @Diskein #3910
3. add allcommands/nocommands to ACL rules by @Lakshyadevelops #3783

What's Changed

• fix: Fix flaky test test_acl_revoke_pub_sub_while_subscribed by @chakaz in #3768
• chore: Forbid replicating a replica by @chakaz in #3779
• fix: GETRANGE params validation by @BorysTheDev in #3781
• fix: bitop do not add dst key if result is empty by @kostasrim in #3751
• chore(acl_family): add allcomands and nocommands by @Lakshyadevelops in #3783
• chore: remove goto statements by @kostasrim in #3791
• chore: ScheduleInternal refactoring by @romange in #3794
• test: update test_noreply_pipeline to prevent false fail by @BorysTheDev in #3801
• fix: bitcount invalid range by @BorysTheDev in #3792
• fix(generic_family): Update indexes in the RESTORE and RENAME commands by @BagritsevichStepan in #3803
• chore: cleanup not used opcodes in replication by @adiholden in #3804
• chore: tune TieredStorageTest.MemoryPressure by @romange in #3805
• fix: make sure dfly_bench reliably connects by @romange in #3802
• chore: reformat listpack according to valkey 8 by @romange in #3810
• fix: empty strings and Nan in the sort command by @BorysTheDev in #3809
• chore: refactor XRead by @dranikpg in #3775
• chore: singlehop XREAD by @dranikpg in #3817
• chore: introduce an optimized integer compare algorithm for lists by @romange in #3813
• chore: add a benchmark comparing lpStringToInt64 to SimpleAtoi by @romange in #3815
• feat(rdb_load): add support for loading huge sets by @andydunstall in #3807
• chore: now it's not needed to allocate quicklistIter on heap by @romange in #3814
• fix(rdb_load): fix loading huge hmaps with ttl by @andydunstall in #3825
• chore: allow config set notify_keyspace_events by @kostasrim in #3790
• chore: do not close connections at the end of pytest by @kostasrim in #3811
• fix(rdb_load): fix appending to an expired key by @andydunstall in #3829
• chore: add logs to test_acl_cat_commands_multi_exec_squash by @kostasrim in #3826
• chore: optimize element removal for integer lists by @romange in #3827
• refactor: remove CmdArgParser::ToUpper() by @BorysTheDev in #3831
• chore: add pending_ops to disk stats by @romange in #3837
• feat(rdb_load): add support for loading huge hmaps and zsets by @andydunstall in #3823
• add dense_set.SetExpiryTime in preparation for fieldexpire by @NegatioN in #3780
• test: more stabe test_bgsave_and_save by @adiholden in #3843
• chore: WrapSds from family_utils.h by @dranikpg in #3818
• chore: add defrag logic for sets by @kostasrim in #3833
• chore: extend RobjWrapper::sz_ to 2^56 by @romange in #3849
• feat(rdb_load): add support for loading huge lists by @andydunstall in #3850
• fix(rdb_load): fix partial reads dropping elements by @andydunstall in #3853
• refactor: refactor fakeredis tests by @cunla in #3852
• chore: simplify CloneBatch code by @romange in #3862
• feat: add HEXPIRE and FIELDEXPIRE by @NegatioN in #3842
• chore: improve performance of ClearInternal by @romange in #3863
• chore: Remove DenseSet::AddOrFindDense and AddSds by @romange in #3864
• chore: Implement AddMany method by @romange in #3866
• chore: ClearInternal now can clear partially by @romange in #3867
• feat(rdb_load): add support for loading huge streams by @andydunstall in #3855
• chore: Refactor AddMany by @romange in #3869
• chore: run memory decommit after snapshot load/save by @kostasrim in #3828
• chore: reduce usage of ToUpper by @romange in #3874
• opt: Optimize AllocationTracker to be efficient when enabled by @chakaz in #3875
• chore: allow rdb version 12 by @kostasrim in #3860
• chore: lock keys for optimistic transactions by @kostasrim in #3865
• fix: Do not publish to connections without context by @chakaz in #3873
• fix: Do not kill Dragonfly on failed DFLY LOAD by @chakaz in #3892
• chore: collection size by @dranikpg in #3844
• chore: skip flaky test_noreply_pipeline by @kostasrim in #3903
• fix(cluster): fix unknown migration error by @andydunstall in #3899
• chrore: parse cgroup v2 by @kostasrim in #3857
• feat(search): STOPWORDS by @dranikpg in #3851
• chore: schedule chains by @romange in #3819
• fix!: fix BITPOS command responses (#3893) by @Diskein in #3910
• fix: prevents Dragonfly from blocking in epoll during snapshotting by @romange in #3911
• fix: macos build by @romange in #3912
• chore(search): Rax TreeMap by @dranikpg in #3909
• chore: Add --allocator_tracker for default tracking by @chakaz in #3901
• chore: relax assertion in test_noreply_pipeline by @kostasrim in #3908
• feat(search): Prefix search* by @dranikpg in #3913
• fix: pre-commit ci workflow by @kostasrim in #3917
• fix: Support replicating Valkey and Redis 7.2 by @chakaz in #3927
• feat(server): use listpack node encoding for list by @adiholden in #3914
• fix: Fix test_flushall_in_full_sync by @chakaz in #3929
• fix: dragonfly_connection should only access the original reply_builder by @romange in #3924
• chore(search): Fix deprecated functions by @dranikpg in #3933

New Contributors

• @Lakshyadevelops made their first contribution in #3783
• @NegatioN made their first contribution in #3780
• @Diskein made their first contribution in #3910

Huge thanks to all the contributors! ❤️

Full Changelog: v1.23.0...v1.24.0

v1.23.2

This is a patch release.
fix: rdb load fix loading plain lists 4959bef

full change log: v1.23.1...v1.23.2

v1.23.1

Dragonfly v1.23.1

This is a patch release.
Update urgency - low.

fix: use less memory when loading huge values: #3829, #3825, #3807
fix: crash in stream code + performance improvements: #3775
fix: crash when using RENAME with search indexes: #3803

Full Changelog: v1.23.0...v1.23.1

v1.23.0

Dragonfly v1.23.0

Some prominent changes include:

• Fix container healthchecks for both privileged and non-privileged environments #3644
• Fix FT.AGGREGATE command (#3657 #3660 )
• Fix regression with the S3 access #3717
• Added ZRANGESTORE command #3757
• introducerss_oom_deny_ratio flag (#3702) - now Dragonfly will reject write commands if its RSS memory usage
  crosses maxmemory*rss_oom_deny_ratio. The default value of this flag is set to 1.25.
• Support for "CLIENT ID" and "CLIENT SETINFO" commands (#3673 and #3672)
• Better memory tracking for JSON objects (#3641)

What's Changed

• fix(docker/healthcheck): run netstat port retreival command as dfly by @Pothulapati in #3647
• test: remove redundant tests by @cunla in #3642
• chore: allow disabling io_uring registered buffers by @romange in #3650
• CmdArgParser improvement by @BorysTheDev in #3633
• fix: deadlock in the cluster migration process by @BorysTheDev in #3653
• fix: fix multi mget exec error message by @BorysTheDev in #3662
• test: fix MC test_expiration by @BorysTheDev in #3663
• chore: add some community suggestions to README by @Niennienzz in #3664
• fix: recursive calls in the allocation tracker by @romange in #3665
• chore: cosmetic changes around Snapshot functions by @romange in #3652
• fix(search_family): Fix FT.AGGREGATE GROUPBY option by @BagritsevichStepan in #3657
• chore: Add CLIENT ID command by @chakaz in #3672
• deprecate RecordsPopper and serialize channel records during push by @romange in #3667
• feat(server): Support CLIENT SETINFO by @chakaz in #3673
• server logs: change script error to warning by @adiholden in #3670
• fix(search_family): Fix LOAD option behavior in the FT.AGGREGATE command by @BagritsevichStepan in #3660
• test: unskip cluster tests and add debug info by @BorysTheDev in #3681
• chore: adjust RdbChannel sizes by @romange in #3676
• chore: add a script that parses allocator tracking logs by @romange in #3687
• chore: limit number of descriptors in the exec map by @romange in #3688
• fix(server): fix replication master deadlock on cancelation flow by @adiholden in #3686
• fix: join for cancel incoming migration by @BorysTheDev in #3692
• fix(json_family): Fix the JSON.SET bug if the path is in legacy mode and is not the root by @BagritsevichStepan in #3693
• fix: test_acl_revoke_pub_sub_while_subscribed by @kostasrim in #3680
• test: fix test_cluster_replication_migration by @BorysTheDev in #3699
• fix: test_replicaof_reject_on_load assert failure by @kostasrim in #3697
• fix(json_family) Add NOESCAPE option to the JSON.GET command by @BagritsevichStepan in #3685
• chore: pull latest helio by @kostasrim in #3700
• feat(core): Support RISCV RVV by @howjmay in #3655
• fix: allow parsing extra spaces on acl files by @kostasrim in #3703
• Update dflycluster slot-migration-status reply by @BorysTheDev in #3707
• chore: add clone benchmark by @romange in #3709
• fix: add default value has_mc_flag field by @BorysTheDev in #3710
• chore: symbolize stack traces in tests upon crash by @romange in #3714
• fix: fix s3 load snapshot by @andydunstall in #3717
• fix(cluster): Do not Pause() replication / migrations by @chakaz in #3716
• chore: add log info on failed commands by @kostasrim in #3694
• chore: enable ReplyGuard in ReplyBuilder2 by @kostasrim in #3705
• tests: fix and enable s3 snapshot test by @andydunstall in #3720
• fix(json_family): Fix JSON.ARRPOP command in legacy mode should not return WRONGTYPE error by @BagritsevichStepan in #3683
• fix(json_family): Separate double and int values during the comparison of the JSON objects by @BagritsevichStepan in #3711
• fix: Fix test test_client_pause_with_replica by @chakaz in #3729
• fix: Fix cluster_mgr.py by @chakaz in #3730
• fix: Fix test_take_over_seeder by @chakaz in #3733
• chore: Remove ReqSerializer by @dranikpg in #3721
• fix(json_family): Fix out of bound ranges for the JSON.ARR* commands by @BagritsevichStepan in #3712
• fix: properly track json memory usage by @kostasrim in #3641
• test: add tests for replication by @BorysTheDev in #3734
• test: unskip test for debugging purpose by @BorysTheDev in #3738
• refactor: use CmdArgParser for XGROUP command by @BorysTheDev in #3739
• test: add test for replication deadlock on replication timeout by @adiholden in #3691
• chore: Switch ports for cluster_mgr_test.py by @chakaz in #3741
• chore: change log level to warning for empty keys by @kostasrim in #3722
• chore: add SetReplies by @dranikpg in #3727
• chore: introduce a Clone function for the dense set by @romange in #3740
• chore: add missing await in test_take_over_seeder by @kostasrim in #3744
• fix: allow the healthcheck run in non-privileged containers as well by @romange in #3731
• fix: add value range check for SETBIT command by @BorysTheDev in #3750
• fix: zset store conclude transaction on error by @andydunstall in #3755
• chore: refactor a lambda function into a named one by @romange in #3753
• chore: fix return on bad status by @dranikpg in #3758
• feat(helm): add hostNetwork, topologySpreadConstraint and clusterIP su… by @katapultcloud in #3389
• feat(server): introduce oom_deny_commands flag by @adiholden in #3718
• feat(server): introduce rss oom limit by @adiholden in #3702
• fix: test deadlock with processing the stdout of sed by @romange in #3735
• chore: add integrity checks to consumer->pel by @romange in #3754
• fix: make snapshotting process more responsive by @romange in #3759
• fix: tune test_replicaof_reject_on_load parameters by @romange in #3762
• chore: logs on assert fail for test_acl_cat_commands_multi_exec_squash by @kostasrim in #3749
• chore: add more error logs around ziplist parsing checks by @romange in #3764
• fix: substr/getrange result for invalid range by @BorysTheDev in #3766
• feat(zset_family): add ZRANGESTORE by @andydunstall in #3757
• fix server: fix last error reply by @adiholden in #3728
• chore: give up on InlinedVector due to spurious warnings with optional by @romange in #3765
• fix: mark pubusb commands as unsupported for cluster by @BorysTheDev in #3767
• chore: some renames + fix a typo in RETURN_ON_BAD_STATUS by @romange in #3763
• fix: setrange with empty value doesn't modify the DB by @BorysTheDev in #3771

New Contributors

• @howjmay made their first contribution in #3655
• @katapultcloud made their first contribution in https://githu...