Dragonfly v1.40.2

v1.40.2
Added 3
  • Add tiered_serialization_inflight_bytes_cap to throttle snapshot serialization when too many outstanding tiered reads exist, preventing unbounded memory growth
  • Export small-bin entries-bytes usage in metrics and INFO
  • Add background repack scan that repacks small-bins buckets with excessive wasted space, disabled by default with tiered_repack_scan_budget_us=0
Changed 2
  • Replace separate write-depth limit with single tiered_max_pending_bytes cap covering both in-flight reads and writes, and export read/write/total usage as metrics
  • Split journal omit optimization into two independent decisions for skipping forced snapshot serialization versus skipping journal write, and add TTL propagation for EXPIRE writes that skip serialization
Fixed 6
  • Values loaded during full sync or RDB-DFS load are no longer incorrectly pushed into the cool-tier cache
  • Bound small-bins traversal with TraverseBySegmentOrder to prevent entries from being skipped or revisited when the table resizes mid-scan
  • Bound DeleteExpiredStep traversal with TraverseBySegmentOrder
  • Fix journal omit breaking on stash bucket
  • Fix memory-accounting bug and journal FLUSHSLOTS from inside flush transaction to avoid replication inconsistency during slot migration
  • Reject memcached keys longer than 250-byte protocol limit to prevent unbounded VALUE reply buffer overflow

From Dragonfly

What's Changed

Tiered storage (disk offload)

  • fix(tiering): Don't cool on load — values loaded during full sync / RDB-DFS load were incorrectly pushed into the cool-tier cache; loading is no longer treated as a candidate for cooling since it isn't client traffic.
  • fix(tiering): Replace the separate write-depth limit with a single tiered_max_pending_bytes cap that covers both in-flight reads and writes, and export read/write/total usage as metrics.
  • fix(tiering): Add tiered_serialization_inflight_bytes_cap — snapshot serialization now throttles/waits when there are too many outstanding tiered reads, preventing unbounded memory growth from delayed entries during snapshotting.
  • fix: Bound small-bins traversal with TraverseBySegmentOrder (same class of fix as DeleteExpiredStep in #8129), preventing entries from being skipped or revisited when the table resizes mid-scan; also exports additional tiering metrics/settings.
  • fix: Export small-bin entries-bytes usage in metrics/INFO.
  • feat(tiering): Add a background "repack scan" that repacks small-bins buckets with excessive wasted space; disabled by default (tiered_repack_scan_budget_us=0).

Replication / journal

  • fix(server): Bound DeleteExpiredStep traversal with TraverseBySegmentOrder (#8129)
  • fix(server): Fix journal omit breaking on stash bucket (#8157)
  • fix(server): Split the "journal omit" optimization into two independent decisions — skipping forced snapshot serialization vs. skipping the journal write — since the former no longer requires the latter; also adds TTL propagation for EXPIRE writes that skip serialization.
  • backport(cluster): Fix memory-accounting bug and journal FLUSHSLOTS from inside the flush transaction, avoiding replication inconsistency during slot migration (#8061, backports #8032)

Protocol

  • fix(facade): Reject memcached keys longer than the 250-byte protocol limit (#8213). Previously, a long key (e.g. written via the Redis port) could be echoed back unbounded in a memcached VALUE reply, overflowing the reply buffer.

Advisory: it is advised to upgrade to this release due to the correctness bug with journaling #8157

View original

Upgraded? How did it go?

Discussion