Dragonfly v1.38.0

v1.38.0
Added 7
  • Top-K command family (TOPK.*) with RedisBloom compatibility and RDB persistence
  • Count-Min Sketch family (CMS.*) with INITBYDIM, INITBYPROB, INCRBY, QUERY, MERGE, and INFO commands with RDB persistence
  • HTTL command to return remaining TTL for specific hash fields
  • VECTOR_RANGE search for radius-based vector similarity queries in FT.SEARCH and FT.AGGREGATE with optional YIELD_DISTANCE_AS
  • FT.AGGREGATE support for FILTER, APPLY, DIALECT, and KNN steps in the aggregation pipeline
  • New Prometheus metrics including pipeline latency histogram, TLS handshake counter, stream access patterns, and pipeline blocking counter
  • AWS credential chain support for EKS Pod Identity and ECS task roles for S3
Changed 6
  • Reduce memory usage by ~26% for TTL workloads by eliminating expire table and embedding expiry in CompactKey via SDS_TTL_TAG
  • KNN prefilter now falls back to brute-force scan when prefilter result set is small
  • Replica expired-key deletion enabled by default with --replica_delete_expired flag allowing replicas to proactively delete expired keys on reads
  • Early TLS filter and TCP_DEFER_ACCEPT drops zombie connections at OS level before allocating fiber and OpenSSL state
  • Memcached io_loop_v2 enabled by default for async I/O with deferred replies
  • Incremental SBF replication flushes Scalable Bloom Filter chunks incrementally to bound peak memory
Fixed 2
  • Fix HNSW index creation failure, logs, and drop
  • Validate stream command arguments to prevent crashes

Dragonfly v1.38.0

This release is a major one with lots of significant improvements and stability fixes. It may behave differently due to changes in its core data structures.
We worked hard to avoid any regressions so please let us know if you observe degradation around item expiry behavior, replication or backups.

Important stability fixes and changes:

Top-K command family (TOPK.*): RedisBloom-compatible heavy-hitter sketch with RDB persistence. (#6920, #6931, #6932, #6950)

Count-Min Sketch family (CMS.*): Full CMS with INITBYDIM, INITBYPROB, INCRBY, QUERY, MERGE, INFO. RDB persistence. (#6867, #6888, #6896, #6897)

~26% memory reduction for TTL workloads: Expire table eliminated, expiry embedded in CompactKey via SDS_TTL_TAG. 10M keys: 900 MB -> 665 MB. (#6923, #6925, #6933)

HTTL command: Returns remaining TTL for specific hash fields. (#6879)

VECTOR_RANGE search: Radius-based vector similarity queries in FT.SEARCH and FT.AGGREGATE for both FLAT and HNSW indexes, with optional YIELD_DISTANCE_AS. (#6880, #6898, #6938)

FT.AGGREGATE: FILTER, APPLY, DIALECT, KNN: Full expression-based filtering, computed fields, dialect selection, and KNN steps in the aggregation pipeline. (#6968, #6982, #7066)

Smarter KNN prefilter: Falls back to brute-force scan when prefilter result set is small. (#6730)

Replica expired-key deletion: Replicas proactively delete expired keys on reads. --replica_delete_expired (default true). (#6985)

New Prometheus metrics: Pipeline latency histogram (dragonfly_pipeline_latency_seconds), TLS handshake counter (tls_handshakes_total), stream access patterns (stream_accesses_total), pipeline blocking counter. (#6688, #7015, #6767, #6685)

EKS Pod Identity for S3: AWS credential chain supports EKS Pod Identity and ECS task roles. (#6917)

Early TLS filter and TCP_DEFER_ACCEPT: Drops zombie connections at OS level before allocating fiber/OpenSSL state. (#6857)

Memcached io_loop_v2 enabled by default*: Async I/O with deferred replies. (#6700)

Incremental SBF replication: Scalable Bloom Filter chunks flushed incrementally, bounding peak memory. (#7034)

What's Changed
Huge thanks to all the contributors! ❤️
New Contributors

Full Changelog: https://github.com/dragonflydb/dragonfly/compare/v1.37.0...v1.38.0

View original

Upgraded? How did it go?

Discussion