Dragonfly v1.40.0

v1.40.0
Added 14
  • Add Cuckoo Filter data type with CF.RESERVE, CF.ADD, CF.ADDNX, CF.EXISTS, CF.MEXISTS, CF.INSERT, CF.INSERTNX, CF.INFO, CF.COUNT, CF.DEL, and CF.COMPACT commands
  • Add RDB serialization support for Cuckoo filters, enabling DUMP/RESTORE and persistence
  • Add EPSILON parameter for HNSW-backed VECTOR_RANGE queries to control overscan and recall near the radius boundary
  • Add support for KNN query attribute block syntax with EF_RUNTIME and YIELD_DISTANCE_AS
  • Add RESET command to reset a connection to a fresh-connection baseline
  • Add Redis-compatible WAIT command to block until replicas acknowledge writes
Changed 5
  • Vector search now stores and computes vectors at native declared width instead of always widening to FLOAT32, reducing memory usage
  • INFO command now includes TLS certificate metadata (subject, issuer, validity dates) when TLS is enabled
  • Enable event-driven V2 IO loop for TLS connections
  • Enable tagged chunk serialization by default during snapshotting and replication
  • GET now returns large raw strings (≥16KB) via zero-copy borrowed view instead of copying
Fixed 2
  • Fix cascading replication correctness when an intermediate replica switches upstream masters under experimental cascaded partial sync
  • Fix crash where an older replica reconnecting to a promoted master could crash the master due to a stale partial-sync index
Security 1
  • Fix CVE-2025-32023 in hyperloglog.c by re-syncing with upstream Valkey

Dragonfly v1.40.0

This is one of our biggest releases yet—both in terms of feature quality, total contributions, and the highest number of external contributors in the last two years.

This release delivers major new functionality alongside a large body of correctness and stability fixes for tiered storage, replication, pub/sub, and the experimental V2 I/O loop.

We added a new data type: Cuckoo Filters (CF.RESERVE, CF.ADD/ADDNX, CF.EXISTS/MEXISTS, CF.INSERT/INSERTNX, CF.INFO/COUNT/DEL/COMPACT, and RDB persistence). This brings Dragonfly to parity with RedisBloom's probabilistic structures alongside expanded Bloom filter (BF.INFO) and HyperLogLog support.

Vector search gains native-width storage across all vector dtypes, EPSILON-tuned HNSW range queries, and EF_RUNTIME control. Meanwhile, a long list of fixes hardens tiered-storage interactions with strings, hashes, lists, bitops, and HLL commands, in addition to several replication-consistency fixes around lazy TTL expiry. Performance work includes lazy lock-table registration, zero-copy GET, and replica-side command batching.

  • New Cuckoo Filter data type with full command set and RDB support.
  • Vector search now stores/computes at native declared width, plus EPSILON and EF_RUNTIME tuning for HNSW.
  • Numerous tiered-storage and replication consistency fixes (HLL, hashes, lists, bitops, hash-field TTL expiry).
Highlights
  • Added a full Cuckoo Filter data type with RDB persistence, bringing RedisBloom parity for probabilistic structures (#7646, #7700).
  • Vector search now stores/computes vectors at native declared width instead of always widening to FLOAT32, reducing memory usage (#7849).
  • Re-synced hyperloglog.c with upstream Valkey, fixing CVE-2025-32023 and adding AVX2/NEON kernels [up to ~29x speedup on dense HLL merges] (#7961).
Commands
  • INFO now includes TLS certificate metadata (subject, issuer, validity dates) when TLS is enabled (#7844).
  • SAVE and BGSAVE now support a HELP subcommand (#7865).
  • JSON.STRAPPEND now supports an optional path argument, defaulting to root (#7868).
  • Implemented the RESET command, resetting a connection to a fresh-connection baseline (#7760).
  • Added Redis-compatible WAIT command, blocking until replicas acknowledge writes (#7454).
  • Added HGETEX and HPEXPIRETIME hash commands (#7717).
  • Added BF.INFO for Bloom filter metadata (#7761).
  • Added BM25STD.NORM and BM25STD.TANH scorers for FT.SEARCH/AGGREGATE/HYBRID (#7692).
Vector Search
  • Vector search (FLAT and HNSW) now stores and computes distances at native declared width (INT8, UINT8, FLOAT16, BFLOAT16, FLOAT32, FLOAT64) (#7849).
  • Added EPSILON parameter for HNSW-backed VECTOR_RANGE queries to control overscan/recall near the radius boundary (#7718).
  • Added support for KNN query attribute block syntax (=>{$EF_RUNTIME: ...; $YIELD_DISTANCE_AS: ...}) (#7708).
  • Fixed filtered KNN vector search to serialize only actual winners instead of every pre-filter match, cutting CPU/memory overhead (#7670).
Cuckoo Filter
  • Added CF.INSERT and CF.INSERTNX for bulk insertion, with optional CAPACITY and NOCREATE (#7719).
  • Added compaction: CF.DEL auto-compacts past 10% deletions, plus a manual CF.COMPACT command (#7702).
  • Added CF.EXISTS and CF.MEXISTS for single/multi-item membership checks (#7682).
  • Added CF.ADD and CF.ADDNX, auto-creating filters with a default capacity of 1024 (#7681).
Probabilistic Data Structures
  • Added RDB serialization support for Cuckoo filters, enabling DUMP/RESTORE and persistence (#7700).
  • Added CF.INFO, CF.COUNT, and CF.DEL for filter metadata, occurrence counts, and item removal (#7691).
  • Added the base Cuckoo Filter data type via CF.RESERVE, with a new CUCKOO_FILTER ACL category (#7646).
Performance
  • Extended parse-in-proactor pipelining optimization to more fiber park spots (shard hops, suspended replies on tiered I/O) (#7916).
  • Added snapshot_egress_limit_bytes to throttle per-shard socket egress during RDB/snapshot saves (#7774).
  • Reworked open-addressing hash container (OAHPair/OAHEntry) with compact variable-width size encoding, reducing fragmentation (#7853).
  • Enabled event-driven V2 IO loop for TLS connections (#7890).
  • Lists loaded from RDB/replication now benefit from ZSTD dictionary compression on load, not just push (#7734).
  • Replica-side Redis replication stream processing now batches commands via DispatchSquashedBatch (#7501).
  • Added write_connection_throttling_sleep_usec to throttle pipelined write batches, plus new batch I/O metrics (#7750).
  • [Total sampled CPU cycles dropped from 64.34B to 59.31B (-7.8%)] Prefetches command contexts before executing squashed pipelined commands (#7796).
  • Optimistic single-shard inline transactions now skip lock-table registration in the common case (#7788).
  • [+9.2% ops/s at pipeline=1, +5.0% at pipeline=10] V2 connection I/O loop drains pending socket data directly from RegisterOnRecv while suspended (#7608).
  • GET now returns large raw strings (≥16KB) via zero-copy borrowed view instead of copying (#7426).
  • Added connection_memory_bytes Prometheus metric tracking active client connection memory (#7550).
  • Wired pipeline squashing into the V2 event-driven dispatch loop via --enable_pipeline_squashing_v2 (#7631).
Replication
  • Fixed cascading replication correctness when an intermediate replica switches upstream masters under experimental cascaded partial sync (#7913).
  • Enabled tagged chunk serialization by default during snapshotting and replication (#7823).
  • RDB loader no longer aborts on EOF with a partially received tagged chunk; now logs a warning and continues (#7660).
  • Fixed a crash where an older replica reconnecting to a promoted master could crash the master due to a stale partial-sync index (#7491).
  • Added a Prometheus metric for replication backlog buffer memory usage (#7644).
Search
  • Applied TEXT field schema weights and query-time $weight attributes to BM25/TF-IDF scoring (#7752).
  • Added parsing support for query-time =>{$WEIGHT} attributes on terms, tag filters, and vector range queries (#7749).
  • Added EF_RUNTIME support for HNSW vector search, actually applying it instead of just parsing it (#7696).
  • Fixed FT.SEARCH quoted TAG values not matching stored values containing backslashes or quotes (#7907).
  • Fixed a race in RestoreStreamer where concurrent Cancel()/Start() could double-unregister a change listener (#7707).
  • Fixed KNN vector search score aliases to properly generate/return implicit score fields (#7604).
  • Fixed FT.SEARCH to return VECTOR fields as raw stored binary buffer instead of textual representation for HASH-stored fields (#7586).
Cluster
  • DFLYCLUSTER SLOT-MIGRATION-STATUS now includes covered slot ranges in its reply (#7911).
  • Fixed a race between Finish() teardown and SyncFb retry loop in slot migration (#7869).
  • Fixed OOM during incoming slot migration not propagating rollback deletion to the target's replica (#7533).
  • Added predefined Helm chart values for enabling Redis-compatible cluster mode (#7123).
Security
  • Fixed ACL DRYRUN gaps: multi-word commands, pub/sub channel checks, and content-dependent key positions (#7919).
  • Fixed ACL DRYRUN to support command arguments for accurate key-pattern permission validation (#7871).
Cloud & Storage
  • Fixed Azure snapshot storage to accept standard Azure Blob HTTPS URLs instead of custom az:// scheme (#7878).
  • Added list_tiering_prefetch_depth to asynchronously prefetch neighboring offloaded list nodes (#7834).
  • Made tiered storage initial backing file size configurable via tiering_disk_storage_initial_size, with 5%-of-capacity growth policy (#7715).
  • Tiered storage now skips offloading keys with remaining TTL below a configurable minimum (tiered_min_ttl_to_offload_ms) (#7624).
Bug fixes
  • Fixed per-slot memory_bytes accounting in cluster mode to correctly track resident RAM across tiered storage operations (#7966).
  • Fixed a read-buffer capacity accounting bug causing client_read_buffer_bytes to leak or double-count (#7951).
  • Fixed two crash/corruption bugs in cluster mode slot operations (FlushSlots and GetSlotStats) and enforced strict db0-only usage (#7960).
  • Fixed blocked XREAD/XREADGROUP being incorrectly woken by lower XADD ids, and a null-pointer crash on plain XREAD (#7959).
  • Fixed blocking XREAD/XREADGROUP on emptied streams returning immediately instead of waiting (#7958).
  • Fixed memory usage reporting for small string keys with TTL to report actual mimalloc block size (#7947).
  • Fixed replication divergence with HEXPIRE, HGETEX, HSETEX, and FIELDEXPIRE on lazily expired fields (#7948).
  • Fixed a crash when Lua scripts call GET on large string values via borrowed replies (#7934).
  • Fixed a replication bug where SHRINK's bucket compaction could leave a deleted key alive forever on a replica (#7938).
  • Fixed a cross-thread data race in command squashing accessing a shared reply builder concurrently (#7927).
  • Fixed EPERM handling in the io_uring support check, falling back to epoll (#7810).
  • Fixed slow Pub/Sub subscribers not being evicted based on per-connection queued-data threshold (#7917).
  • Fixed a regression where UNSUBSCRIBE could jump ahead of queued pub/sub messages (#7909).
  • Fixed the V2 dispatch queue loop not setting async_op_start_cycle_, hiding timed-out async operations (#7902).
  • Dragonfly now force-closes blocked Pub/Sub subscribers exceeding a configurable soft memory limit (#7889).
  • Fixed a bug where a dropped checkpoint message leaked the DispatchTracker counter, hanging REPLTAKEOVER/CLIENT PAUSE/shutdown (#7862).
  • Fixed a race condition in REPLTAKEOVER where master-check and replica pointer check weren't atomic (#7870).
  • Fixed a use-after-free bug garbling long strings (stream IDs, sortkeys, scan cursors) sent by reference (#7874).
  • Fixed a bug where a blocked XREAD/XREADGROUP with COUNT ignored the COUNT limit on wake-up (#7872).
  • RDB loading now skips keys with unsupported module types instead of aborting the whole load (#7912).
  • Fixed a bug causing Dragonfly to SIGABRT on invalid startup flags instead of exiting cleanly (#7843).
  • Fixed crashes/incorrect behavior when commands accessed offloaded string values (DEBUG OBJECT, INCRBYFLOAT, memcached APPEND/PREPEND, JSON.GET) (#7836).
  • Fixed a race where heartbeat expiry/eviction could run while an optimistic transaction was suspended (#7826).
  • Fixed crashes in HLL commands when values were offloaded to tiered storage, plus a pipeline-squashing race corrupting results (#7820).
  • Fixed a race in the V2 (io_uring) connection loop allowing CLIENT PAUSE/takeover/migration/shutdown to proceed early (#7805).
  • Fixed bitops commands (GETBIT, SETBIT, BITCOUNT, BITPOS, BITOP, BITFIELD) crashing on tiered string values (#7806).
  • Fixed a bug where stopping MONITOR mode didn't decrement the subscriptions counter (#7777).
  • Fixed the V2 IoLoopV2 path not tracking in-flight command dispatches, breaking CLIENT PAUSE/takeover/migration (#7764).
  • Fixed buffered replies remaining undelivered when a subsequent pipelined command blocked in the V2 path (#7733).
  • Fixed missing bookkeeping (command counters, idle timestamp) in the V2 connection batch dispatch path (#7726).
  • Fixed a bug where a bare GETEX (no expiry options) was incorrectly replicated (#7701).
  • Fixed a stats bug where io_uring multishot recv failed to update read counters (#7680).
  • Fixed dragonfly_commands_processed_total always reporting 0 (#7667).
  • Fixed a crash in Lua scripting when redis.call() received a non-string/number command name (#7655).
  • Fixed a QList bug where inserting before the first element of a full head node misplaced the element (#7640).
  • Fixed tiered list bugs: node deletion promoting offloaded nodes without materializing (#7638), stale entry pointers after restore (#7636).
  • Fixed a bug where MOVEing a key between databases left stale search index and memory-usage entries (#7637).
  • Fixed a replication bug where an older replica reconnecting to a promoted master could crash it (#7491).
  • Fixed a crash when admin control commands appeared inside a squashed pipeline (#7621).
  • Fixed a parsing bug where malformed RESP was silently treated as needing more input instead of a protocol error (#7615).
  • Restored the experimental_io_loop_v2 flag after accidental removal (#7607).
  • Fixed keyspace-notification delivery running inside a fiber-atomic section during heartbeat expiry (#7540).
  • Re-enabled fiber yielding during long-running container iteration commands (#7449).
  • Fixed BITFIELD, BITOP, and SETBIT accepting out-of-range bit offsets, risking overflow/crashes (#7513).
Huge thanks to all the contributors! ❤️
New Contributors
What's Changed

Full Changelog: https://github.com/dragonflydb/dragonfly/compare/v1.39.0...v1.40.0

View original

Upgraded? How did it go?

Discussion