NATS

Changelog

Refer to the 2.14 Upgrade Guide for backwards compatibility notes with 2.12.x. Please note that the 2.13.x version was skipped.

Go Version

• 1.26.8

Dependencies

• golang.org/x/crypto v0.57.0 (#8597)
• golang.org/x/sys v0.48.0 (#8597)
• golang.org/x/time v0.16.0 (#8597)

Fixed

JetStream

• Fixed a data race between deduplication tracking and stream config updates (#8598)
• Fixed a potential nil consumer info when responding to consumer reset requests (#8598)
• Stream catchups will no longer run for removed or re-synced peers (#8598)
• Fixed a race condition that could leave ghost consumer assignments when deleting consumers (#8598)
• Pausing a consumer with pending changes no longer cancels out the in-flight assignment (#8598)
• Stepping down a consumer leader with pending changes no longer fails (#8598)
• Switching a stream to interest retention policy with no consumers now correctly cleans up messages with no interest (#8598)
• Fixed an issue where removed consumers were incorrectly matched without checking the group name (#8598)
• A number of stream and consumer endpoints will no longer report the wrong state based on in-flight proposals (#8598)

Complete Changes

https://github.com/nats-io/nats-server/compare/v2.14.7-RC.1...v2.14.7-RC.2

Changelog

Refer to the 2.14 Upgrade Guide for backwards compatibility notes with 2.12.x. Please note that the 2.13.x version was skipped.

Go Version

• 1.26.8 (#8551)

Dependencies

• github.com/antithesishq/antithesis-sdk-go v0.8.0-default-no-op (#8562)
• github.com/klauspost/compress v1.20.0 (#8562)
• golang.org/x/crypto v0.56.0 (#8562)

Added

JetStream

• Compatibility for the new desired state metalayer, introduced in v2.15, to support safe upgrades and downgrades (#8554)

Improved

General

• The fastrand and math/rand packages have been replaced with math/rand/v2 (#8538, #8557)
• Reduced lock contention that could arise from tracking event IDs or closed connections (#8533)

JetStream

• When syncing deletes from a snapshot, block removals are now aggregated, reducing pending calculations (#8545)
• Message size checks are now performed before more expensive header checks (#8561)
• Pending calculations are now considerably faster when the max messages per subject limit is 1 (#8567)

Fixed

General

• The {{left}} and {{right}} mapping functions now work correctly in account mappings and source/mirror transforms (#8568)

JetStream

• A potential underflow when calculating deletes in skip messages has been fixed (#8546)
• Fixed an issue where catchup signals to the metalayer are blocked, which could stall stream or cluster management operations until the node is restarted (#8555)
• Consumer removal is now handled correctly after a recreate during metalayer recovery at startup (#8556)
• Fixed a potential panic that could happen when filestore block caches were released during message removal (#8548)
• The last dedupe message ID is now restored correctly if the last message was outside of the window after restart (#8564)
• An unreachable send snapshot condition has been removed from the stream monitor (#8565)
• Pull request expiry is now counted as activity when inactive_threshold is set (#8575)
• Purges and roll-ups on multi-filter subject consumers will no longer advance past unrelated subjects (#8572)
• Filestore compression header checking is now extended to check for compatible compression values and for the presence of correct or incorrect first record checksums, avoiding collisions at the beginning of a message block (#8581)
• JetStream usage from peer-removed nodes is now removed from the remaining cluster node accounting correctly (#8570)

Complete Changes

https://github.com/nats-io/nats-server/compare/v2.14.6...v2.14.7-RC.1

Changelog

This release also contains all changes up to and including v2.14.6.

Go Version

• 1.27.1 (#8551)

Added

JetStream

• Desired state metalayer (#8432, #8437, #8439, #8443, #8452, #8460, #8476, #8576)
  • A new desired state reconciliation engine for streams and consumers considerably improves the safety and reliability of asset moves and scales
  • Updating the placement or replication factor midway through an existing scale/move operation, cancelling an in-flight scale/move, or peer-removing is now much safer
• Cancel stream move endpoint (#8476)
  • An in-flight scale/move operation on a stream can be cancelled and the asset returned to its original placement using the new $JS.API.STREAM.CANCEL_MOVE endpoint
• Evacuate endpoints (#8443)
  • The new $JS.API.SERVER.EVACUATE endpoint can safely evacuate streams, and any consumers on those streams, from a node
  • The new $JS.API.STREAM.PEER.EVACUATE.* endpoint can safely evacuate a peer, and any consumers on that peer, from a stream
  • These operations combined allow for maintenance operations with full transfer of data and state without having to peer-remove first
• Metalayer rescue for disaster recovery (#8408)
  • The new $JS.API.META.RESCUE endpoint can temporarily lower the quorum-needed for the metalayer, in order to facilitate the recovery of a system where nodes have been permanently lost and the metalayer cannot reach quorum
• Stream backup and restore v2 (#7882)
  • A new backup format for stream snapshots which reads out per-message rather than per-block
  • The new backup format also now correctly includes non-replicated consumers on follower nodes
• Detect source stream recreation (#8384)
  • The server now detects a source stream being recreated, restarting the sourcing back from the beginning, ensuring new messages are sourced
• Stream source indexing (#8282, #8516, #8283)
  • Restarts and leader changes previously required expensive backward scans through the stream to find the last sourced indices. These are now persisted in an index for instant lookup.
• Domain-prefixed JS API in system account (#8429)
  • When a system account is bridged between the hub and a leaf node, the domain-prefixed JS API can now be used to operate on the leaf node while directly connected to the hub

Changed

JetStream

• Streams now have a default limit of 1000 consumers, unless max_consumers is specified in the stream config or account limits (#8337, #8566)
  • Applications planning to use a large number of consumers on a stream should configure max_consumers in the stream config or account limits to an appropriately high number
  • The server-wide default of 1000 can be overridden by setting default_max_consumers in the JetStream limits, or can be disabled by setting to -1
  • This limit does not affect existing consumers and will not result in consumer deletions, only the creation of new consumers is prevented
• Sync changes for replicated streams (#8447)
  • When the sync_interval is set to always, replicated streams now sync their WAL entries but no longer sync upper stream layer writes
  • The synced log allows the stream to recover safely, but removing unnecessary syncs from the upper stream layer dramatically improves performance
  • Non-replicated R1 streams are not affected by this change and will continue to sync at the stream layer as normal
• The js_raft_delete_range feature is now enabled by default (#8520)

Leafnodes

• The no_advertise option no longer includes the hub's own listener address (#8451)

Improved

JetStream

• Shorter wait for durable source/mirror consumer resets (#8323)
  • When using durable consumers for sourcing or mirroring, the heartbeats now short-circuit the recreation backoff

Fixed

Leafnodes

• Leafnode isolation using request_isolation, or with hub-enforced isolate, now works correctly across cluster nodes (#8504)

Complete Changes

https://github.com/nats-io/nats-server/compare/v2.14.6...v2.15.0-RC.1

Changelog

Refer to the 2.14 Upgrade Guide for backwards compatibility notes with 2.12.x. Please note that the 2.13.x version was skipped.

Go Version

• 1.26.7

Improved

General

• Removal from service import response maps is now constant-time (#8463)
• The client code no longer resets the deadline more times than necessary when flushing client buffers (#8513)
• The client write buffers are now freed correctly when the working buffer exceeds a rational size (#8515)

JetStream

• Stream reads are now isolated with a separate lock, which improves the performance of direct gets (#8486)
• Improved the performance of looking up matching sequence numbers for multi-subject gets (#8486)
• Reduced the number of heap escapes in subject tree matching (#8526)

Fixed

General

• WebSocket client buffers are recycled more aggressively when reallocating into larger buffers (#8518)
• Fixed missing routes after a reconnect when gossiped URLs were rejected (#8527)

JetStream

• The filestore block sync no longer incorrectly skips blocks that were compacted during the same pass (#8456)
• The filestore block sync now performs a single directory sync upon completion rather than once per compacted block (#8461)
• The filestore I/O semaphore now covers block sync operations (#8462)
• Stream config updates with replicas greater than 1 are now rejected on non-clustered systems (#8464)
• Fixed some issues that could prevent stream snapshots from taking place on a clean shutdown (#8465)
• The filestore inline compaction no longer ignores sync_interval: always (#8475)
• The stream created time is now preserved after recovery on a standalone server (#8471, contributed by @Green7)
• Fixed a data race that could apply when accessing the batch ID (#8369)
• Fixed a data race when reading the consumer direct or sourcing status (#8478, contributed by @nberlee)
• Prevent the consumer direct or sourcing status from being changed with a consumer update (#8478, contributed by @nberlee)
• Fixed an issue where the server could fail to distinguish consumer tiers when enforcing limits (#8484)
• Fixed a flow control problem where replicated consumers could get stuck after a leader change (#8488)
• Fixed various consumer create issues that could destroy the state of an existing consumer with the same name (#8491)
• Internal message delete proposals are no longer counted in the JS API statistics (#8502)
• Dynamic filestore reservations no longer incorrectly shrink based on used storage after restarts (#8503)
• Fixed a signalling issue that could stall Raft catchups (#8501)
• The pending append entry cache in-memory is now bounded by size as well as entry count (#8501)
• Fixed an issue where a stale snapshot from a previous Raft group could be replayed if the group name was unchanged (#8501)
• Consumer delivery counts will no longer underflow below zero (#8512, contributed by @no-hup)
• AckFlowControl consumers from a work queue stream will no longer incorrectly ack messages outside of the consumer filter (#8431, #8528)

Complete Changes

https://github.com/nats-io/nats-server/compare/v2.14.5...v2.14.6

Changelog

Refer to the 2.14 Upgrade Guide for backwards compatibility notes with 2.12.x. Please note that the 2.13.x version was skipped.

Go Version

• 1.26.7

Improved

JetStream

• Reduced the number of heap escapes in subject tree matching (#8526)

Fixed

JetStream

• Fixed a signalling issue that could stall Raft catchups (#8501)
• The pending append entry cache in-memory is now bounded by size as well as entry count (#8501)
• Fixed an issue where a stale snapshot from a previous Raft group could be replayed if the group name was unchanged (#8501)
• Consumer delivery counts will no longer underflow below zero (#8512, contributed by @no-hup)

Complete Changes

https://github.com/nats-io/nats-server/compare/v2.14.6-RC.1...v2.14.6-RC.2

Changelog

Refer to the 2.14 Upgrade Guide for backwards compatibility notes with 2.12.x. Please note that the 2.13.x version was skipped.

Go Version

• 1.26.7

Improved

General

• Removal from service import response maps is now constant-time (#8463)
• The client code no longer resets the deadline more times than necessary when flushing client buffers (#8513)
• The client write buffers are now freed correctly when the working buffer exceeds a rational size (#8515)

JetStream

• Stream reads are now isolated with a separate lock, which improves the performance of direct gets (#8486)
• Improved the performance of looking up matching sequence numbers for multi-subject gets (#8486)

Fixed

General

• WebSocket client buffers are recycled more aggressively when reallocating into larger buffers (#8518)

JetStream

• The filestore block sync no longer incorrectly skips blocks that were compacted during the same pass (#8456)
• The filestore block sync now performs a single directory sync upon completion rather than once per compacted block (#8461)
• The filestore I/O semaphore now covers block sync operations (#8462)
• Stream config updates with replicas greater than 1 are now rejected on non-clustered systems (#8464)
• Fixed some issues that could prevent stream snapshots from taking place on a clean shutdown (#8465)
• The filestore inline compaction no longer ignores sync_interval: always (#8475)
• The stream created time is now preserved after recovery on a standalone server (#8471, contributed by @Green7)
• Fixed a data race that could apply when accessing the batch ID (#8369)
• Fixed a data race when reading the consumer direct or sourcing status (#8478, contributed by @nberlee)
• Prevent the consumer direct or sourcing status from being changed with a consumer update (#8478, contributed by @nberlee)
• Fixed an issue where the server could fail to distinguish consumer tiers when enforcing limits (#8484)
• Fixed a flow control problem where replicated consumers could get stuck after a leader change (#8488)
• Fixed various consumer create issues that could destroy the state of an existing consumer with the same name (#8491)
• Internal message delete proposals are no longer counted in the JS API statistics (#8502)
• Dynamic filestore reservations no longer incorrectly shrink based on used storage after restarts (#8503)

Complete Changes

https://github.com/nats-io/nats-server/compare/v2.14.5...v2.14.6-RC.1

Changelog

[!IMPORTANT] This is a preview release for the upcoming 2.15 series. This is not a formal release candidate and is still undergoing testing, but we would welcome feedback.

This release also contains all changes up to and including v2.14.5.

Go Version

• 1.27.0

Added

JetStream

• Desired state metalayer (#8432, #8437, #8439, #8443, #8452, #8460, #8476)
  • A new desired state reconciliation engine for streams and consumers considerably improves the safety and reliability of asset moves and scales
  • Updating the placement or replication factor midway through an existing scale/move operation, cancelling an in-flight scale/move, or peer-removing is now much safer
• Cancel stream move endpoint (#8476)
  • An in-flight scale/move operation on a stream can be cancelled and the asset returned to its original placement using the new $JS.API.STREAM.CANCEL_MOVE endpoint
• Evacuate endpoints (#8443)
  • The new $JS.API.SERVER.EVACUATE endpoint can safely evacuate streams, and any consumers on those streams, from a node
  • The new $JS.API.STREAM.PEER.EVACUATE.* endpoint can safely evacuate a peer, and any consumers on that peer, from a stream
  • These operations combined allow for maintenance operations with full transfer of data and state without having to peer-remove first
• Metalayer rescue for disaster recovery (#8408)
  • The new $JS.API.META.RESCUE endpoint can temporarily lower the quorum-needed for the metalayer, in order to facilitate the recovery of a system where nodes have been permanently lost and the metalayer cannot reach quorum
• Stream backup and restore v2 (#7882)
  • A new backup format for stream snapshots which reads out per-message rather than per-block
  • The new backup format also now correctly includes non-replicated consumers on follower nodes
• Detect source stream recreation (#8384)
  • The server now detects a source stream being recreated, restarting the sourcing back from the beginning, ensuring new messages are sourced
• Stream source indexing (#8282)
  • Restarts and leader changes previously required expensive backward scans through the stream to find the last sourced indices. These are now persisted in an index for instant lookup.
• Domain-prefixed JS API in system account (#8429)
  • When a system account is bridged between the hub and a leaf node, the domain-prefixed JS API can now be used to operate on the leaf node while directly connected to the hub

Changed

JetStream

• Sync changes for replicated streams (#8447)
  • When the sync_interval is set to always, replicated streams now sync their WAL entries but no longer sync upper stream layer writes
  • The synced log allows the stream to recover safely, but removing unnecessary syncs from the upper stream layer dramatically improves performance
  • Non-replicated R1 streams are not affected by this change and will continue to sync at the stream layer as normal

Improved

JetStream

• Shorter wait for durable source/mirror consumer resets (#8323)
  • When using durable consumers for sourcing or mirroring, the heartbeats now short-circuit the recreation backoff

Complete Changes

https://github.com/nats-io/nats-server/compare/v2.14.5...v2.15.0-preview.1

Changelog

Refer to the 2.14 Upgrade Guide for backwards compatibility notes with 2.12.x. Please note that the 2.13.x version was skipped.

Go Version

• 1.26.5

Dependencies

• github.com/klauspost/compress v1.19.2 (#8459)
• golang.org/x/crypto v0.55.0 (#8459)

Added

Leafnodes

• New dial_timeout option can be specified in the leafnode config block or for specific remotes in the configuration, allowing it to be increased above the default 1 second for high-latency links (#8427)

Fixed

General

• Fixed a deadlock that could occur in the logger when a close error occurs (#8430)

JetStream

• Fix a bug that could result in potential data loss when handling idempotent stream creates when an offline node catches up from a metalayer snapshot, caused by an incorrect update to the create time in the stream assignment (#8449)

Complete Changes

https://github.com/nats-io/nats-server/compare/v2.14.4...v2.14.5

Changelog

Go Version

• 1.25.12

Dependencies

• github.com/klauspost/compress v1.19.2 (#8459)
• golang.org/x/crypto v0.55.0 (#8459)

Fixed

General

• Fixed a deadlock that could occur in the logger when a close error occurs (#8430)

JetStream

• Fix a bug that could result in potential data loss when handling idempotent stream creates when an offline node catches up from a metalayer snapshot, caused by an incorrect update to the create time in the stream assignment (#8449)

Complete Changes

https://github.com/nats-io/nats-server/compare/v2.12.14...v2.12.15

Changelog

Refer to the 2.14 Upgrade Guide for backwards compatibility notes with 2.12.x. Please note that the 2.13.x version was skipped.

Go Version

• 1.26.5 (#8435)

Dependencies

• github.com/klauspost/compress v1.19.0 (#8385)
• golang.org/x/crypto v0.54.0 (#8385)
• golang.org/x/sys v0.47.0 (#8385)
• github.com/antithesishq/antithesis-sdk-go v0.7.2-default-no-op (#8385)

Improved

JetStream

• The Raft transport layer has been decoupled, improves testing but does not change server behaviour (#8181)
• The disk concurrency semaphore has been increased to 4096 slots, up from the previous CPU-scaled count (#8336)
• The disk concurrency semaphore can now be configured with the max_concurrent_io option in the jetstream config block (#8336)
• Filestore underlying block cache buffers are now recycled to the pool when the weak reference is collected by the GC, which should smooth out memory usage with some usage patterns (#8395)
• Calculating and looking up sequences in delete maps for file-backed streams with large numbers of interior deletes is now faster and holds locks for less time (#8403)
• Inserts, iterations and deletes in AVL sequence sets are now faster in many cases, which speeds up the tracking of interior deletes (#8406)
• Stream snapshots now attempt to determine the correct encode buffer size up front, avoiding many unnecessary allocations on streams with large numbers of interior deletes (#8405)
• Reduced memory usage of the structure that is used to track subjects within a stream (#8412)

Fixed

General

• allow_non_tls will no longer log an incorrect message at startup claiming that TLS is required (#8420)
• Combining no_auth_user with auth callouts will no longer skip authentication checks when no CONNECT message is sent
• JWT validation no longer crashes the server with whitespace-only permissions
• Several paths that enforce the permissions of queue subscriptions no longer treat the whole permission as a subject literal
• Several JetStream and MQTT endpoints now correctly guard against null values in JSON
• Fixed an authentication bypass with TLS verify_and_map authenticating users with blank passwords

Monitoring

• The healthz endpoint will now skip and no longer report on expired JWT accounts (#8379, thanks to @ByapakSigdel)
• The varz endpoint will now correctly report JetStream limits after they were changed via a config reload (#8394)

JetStream

• Malformed cluster replicated acks or delivered updates are now correctly rejected by the decoder (#8284, thanks to @uwezkhan)
• Malformed cluster replicated skip or reset updates are now correctly rejected by the decoder (#8345, thanks to @uwezkhan)
• Empty cluster replicated entries are now correctly ignored (#8347, thanks to @uwezkhan)
• Decoded AVL sequence set node counts are now validated correctly on 32-bit systems (#8355, thanks to @uwezkhan)
• Stale error responses on source or mirror creation are now dropped by recreating the subscription (#8356)
• Oversized cluster replicated messages are now validated correctly on 32-bit systems (#8357, thanks to @uwezkhan)
• Raft elections now correctly ignore votes from removed peers (#8353)
• Filestore blocks with unsynced or truncated key files are now removed and counted as lost data instead of failing to recover altogether (#8365)
• Filestore encryption key files are now synced to disk more aggressively (#8366)
• Raft now handles the append entry iterator returning no more entries correctly (#8372)
• Fixed string ownership when handling the expected last sequence per subject in a batch (#8377)
• Fixed a race condition between concurrent message removals via limits that could unexpectedly disable writes into a filestore (#8378)
• Fixed a bug in the filestore which could prevent some filestore block cache references from being weakened correctly, which could result in unexpected memory usage and GC pressure (#8380)
• Attempting to update the consumer storage type now correctly returns an error (#8382)
• Stream publish checks will now correctly reject messages that exceed the maximum store size before proposal (#8389)
• Fixed a race condition between storing messages in the filestore and block compactions (#8400)
• The filestore no longer skips sparse delete blocks when processing deletes from a snapshot (#8404)
• The consumer reset endpoint responses are no longer dropped when sent through a service import (#8407)
• Creating a clustered consumer immediately after creating a clustered stream should no longer respond with a stream not found error (#8410)
• Raft proposals now require the term to be passed down from JetStream, preventing situations where stale proposals from a previous term could make changes in a new term after a fast election (#8370)
• Replicated streams that were recreated while a node was down are no longer treated as an update by a returning node processing a snapshot, avoiding stale Raft groups from continuing to run and unexpected behaviour with consumers (#8413)
• Stream snapshot endpoints now more strictly check the reply subject for validity

MQTT

• Packet identifiers for QoS1 and QoS2 are now issued by a monotonic counter, avoiding accidental ID reuse (#8358, thanks to @nberlee)
• Pending QoS1 and QoS2 deliveries should no longer leak when the subscription is downgraded to QoS0 (#8359, thanks to @nberlee)
• QoS2 messages released on a resumed sessions should no longer lose their QoS or packet ID (#8414)
• MQTT clients can no longer subscribe to $MQTT.> subjects, closing a potential permission bypass

Complete Changes

https://github.com/nats-io/nats-server/compare/v2.14.3...v2.14.4

Changelog

Go Version

• 1.25.12 (#8435)

Dependencies

• github.com/klauspost/compress v1.19.0 (#8385)
• golang.org/x/crypto v0.54.0 (#8385)
• golang.org/x/sys v0.47.0 (#8385)
• github.com/antithesishq/antithesis-sdk-go v0.7.2-default-no-op (#8385)

Improved

JetStream

• The Raft transport layer has been decoupled, improves testing but does not change server behaviour (#8181)
• The disk concurrency semaphore has been increased to 4096 slots, up from the previous CPU-scaled count (#8336)
• The disk concurrency semaphore can now be configured with the max_concurrent_io option in the jetstream config block (#8336)
• Filestore underlying block cache buffers are now recycled to the pool when the weak reference is collected by the GC, which should smooth out memory usage with some usage patterns (#8395)
• Inserts, iterations and deletes in AVL sequence sets are now faster in many cases, which speeds up the tracking of interior deletes (#8406)
• Stream snapshots now attempt to determine the correct encode buffer size up front, avoiding many unnecessary allocations on streams with large numbers of interior deletes (#8405)
• Reduced memory usage of the structure that is used to track subjects within a stream (#8412)

Fixed

General

• allow_non_tls will no longer log an incorrect message at startup claiming that TLS is required (#8420)
• Combining no_auth_user with auth callouts will no longer skip authentication checks when no CONNECT message is sent
• JWT validation no longer crashes the server with whitespace-only permissions
• Several paths that enforce the permissions of queue subscriptions no longer treat the whole permission as a subject literal
• Several JetStream and MQTT endpoints now correctly guard against null values in JSON
• Fixed an authentication bypass with TLS verify_and_map authenticating users with blank passwords

Monitoring

• The healthz endpoint will now skip and no longer report on expired JWT accounts (#8379, thanks to @ByapakSigdel)
• The varz endpoint will now correctly report JetStream limits after they were changed via a config reload (#8394)

JetStream

• Malformed cluster replicated acks or delivered updates are now correctly rejected by the decoder (#8284, thanks to @uwezkhan)
• Malformed cluster replicated skip or reset updates are now correctly rejected by the decoder (#8345, thanks to @uwezkhan)
• Empty cluster replicated entries are now correctly ignored (#8347, thanks to @uwezkhan)
• Decoded AVL sequence set node counts are now validated correctly on 32-bit systems (#8355, thanks to @uwezkhan)
• Oversized cluster replicated messages are now validated correctly on 32-bit systems (#8357, thanks to @uwezkhan)
• Raft elections now correctly ignore votes from removed peers (#8353)
• Filestore encryption key files are now synced to disk more aggressively (#8366)
• Raft now handles the append entry iterator returning no more entries correctly (#8372)
• Fixed a bug in the filestore which could prevent some filestore block cache references from being weakened correctly, which could result in unexpected memory usage and GC pressure (#8380)
• Attempting to update the consumer storage type now correctly returns an error (#8382)
• Stream publish checks will now correctly reject messages that exceed the maximum store size before proposal (#8389)
• Creating a clustered consumer immediately after creating a clustered stream should no longer respond with a stream not found error (#8410)
• Replicated streams that were recreated while a node was down are no longer treated as an update by a returning node processing a snapshot, avoiding stale Raft groups from continuing to run and unexpected behaviour with consumers (#8413)
• Stream snapshot endpoints now more strictly check the reply subject for validity

MQTT

• Packet identifiers for QoS1 and QoS2 are now issued by a monotonic counter, avoiding accidental ID reuse (#8358, thanks to @nberlee)
• Pending QoS1 and QoS2 deliveries should no longer leak when the subscription is downgraded to QoS0 (#8359, thanks to @nberlee)
• QoS2 messages released on a resumed sessions should no longer lose their QoS or packet ID (#8414)

Complete Changes

https://github.com/nats-io/nats-server/compare/v2.12.12...v2.12.14

Changelog

Refer to the 2.14 Upgrade Guide for backwards compatibility notes with 2.12.x. Please note that the 2.13.x version was skipped.

Go Version

• 1.26.4 (#8281)

Dependencies

• golang.org/x/crypto v0.53.0 (#8297)
• golang.org/x/sys v0.46.0 (#8297)
• github.com/nats-io/jwt/v2 v2.8.2
• github.com/nats-io/nkeys v0.4.16

Improved

General

• Per-connection log lines that could be noisy in normal operation have been demoted to debug level (#8289)
• Writer options are now applied consistently when using the s2_fast compression mode (#8047)

JetStream

• Stream and consumer assignment handling has been refactored for more consistent migration and info behavior (#8262)
• Meta, stream and consumer write errors are now registered more consistently for health and recovery handling (#8293)

Removed

Monitoring

• JSONP callback support has been removed from monitoring endpoints

Fixed

General

• Long-running reconnect and OCSP loops no longer retain unused timers, reducing memory pressure over time (#8204)
• Inherited JWT default permissions are now refreshed when account claims are updated (#8276)
• External auth configuration is now cleared correctly when account claims are updated (#8275)
• PROXY protocol detection, TLS sniffing with allow_non_tls and PROXY v1 address-family parsing have been fixed (#8302)
• A race in gateway CONNECT handling has been fixed (#8306)
• Trusted proxy tracking no longer leaks closed clients during concurrent updates (#8307)
• Service import replies can now be delivered across cluster routes (#8317)
• Message tracing now works correctly with service imports and exports
• Several panic, fatal and data race conditions in authentication, routing, monitoring and clustered request handling have been fixed
• NoAuthUser now checks connection restrictions
• Leaf connections no longer bypass Nats-Trace-Dest publish permission checks
• CONNZ and SUBSZ pagination now guard against Offset and Limit integer overflow panics
• Fixed a nil pointer panic when starting up when the resolver parent directory is missing (#8329)

MQTT

• Partial CONNECT packets can no longer exhaust pre-authentication memory
• PUBLISH remaining-length underflow no longer causes a server panic
• Subscriptions to internal $MQTT.deliver.pubrel subjects are now rejected
• Subscribe deny rules are now enforced on retained message and QoS replay paths
• WebSocket /mqtt upgrades no longer panic when MQTT is disabled

Monitoring

• JetStream remote usage updates no longer panic on length integer overflow

JetStream

• A data race on the cluster meta node during JetStream shutdown has been fixed (#8260)
• Meta proposal inflight tracking is now kept consistent during stream moves and related operations (#8261)
• Stream catchup is no longer skipped when limits are exceeded, preventing possible stream desync (#8265)
• Malformed TTL and schedule state is now rejected during decode (#8269)
• Zero consumer limits are now treated as unlimited during stream updates (#8286)
• Raft nodes no longer participate in voting or candidacy after write errors (#8290)
• Raft checkpoint handling now aborts if the node is closed (#8296)
• Raft ApplyCommit now handles the post-snapshot index correctly (#8321)
• Consumer ack subscriptions now match correctly when consumer names contain % (#8301)
• Observer state is now cleared correctly during js_cluster_migrate when a leaf remote is removed (#8304)
• Atomic batch end-of-batch max-size checks and R1 message rewrites have been fixed (#8305)
• Schedule drift, failed fast batch commits with gapOk and stale /varz leaf remote state have been fixed (#8308)
• Peer state decoding now bounds peer ID reads to the buffer length (#8310)
• Counter stream staging no longer corrupts the committed running total (#8311)
• Filestore compaction no longer corrupts compressed or encrypted blocks (#8312)
• Memory store NumPending no longer overcounts for DeliverLastPerSubject consumers (#8313)
• Consumer inactive-delete grace period handling and pull request MaxBytes budgeting have been fixed (#8314)
• MultiLastSeqs no longer reorders stream config subjects through filterIsAll handling (#8315)
• Meta recovery snapshots no longer leave phantom streams or consumers behind (#8324)
• Skipped messages last time no longer violates ordering that could lead to issues with starting by time (#8237)
• Raft now reverts uncommitted membership changes correctly when truncating or snapshotting (#8332)

Credits

While CVE advisory notices are credited individually, a number of fixes in this release were the result of non-CVE reports from the following contributors:

• Koda Reef
• @Emin-ACIKGOZ
• @0xVijay
• Yaohui Wang
• @alanturing881

Complete Changes

https://github.com/nats-io/nats-server/compare/v2.14.2...v2.14.3

Changelog

Go Version

• 1.25.11

Dependencies

• golang.org/x/crypto v0.53.0 (#8297)
• golang.org/x/sys v0.46.0 (#8297)
• github.com/nats-io/jwt/v2 v2.8.2
• github.com/nats-io/nkeys v0.4.16

Improved

General

• Per-connection log lines that could be noisy in normal operation have been demoted to debug level (#8289)
• Writer options are now applied consistently when using the s2_fast compression mode (#8047)

JetStream

• Stream and consumer assignment handling has been refactored for more consistent migration and info behavior (#8262)
• Meta, stream and consumer write errors are now registered more consistently for health and recovery handling (#8293)

Removed

Monitoring

• JSONP callback support has been removed from monitoring endpoints

Fixed

General

• Inherited JWT default permissions are now refreshed when account claims are updated (#8276)
• External auth configuration is now cleared correctly when account claims are updated (#8275)
• PROXY protocol detection, TLS sniffing with allow_non_tls and PROXY v1 address-family parsing have been fixed (#8302)
• A race in gateway CONNECT handling has been fixed (#8306)
• Trusted proxy tracking no longer leaks closed clients during concurrent updates (#8307)
• Service import replies can now be delivered across cluster routes (#8317)
• Message tracing now works correctly with service imports and exports
• Several panic, fatal and data race conditions in authentication, routing, monitoring and clustered request handling have been fixed
• NoAuthUser now checks connection restrictions
• CONNZ and SUBSZ pagination now guard against Offset and Limit integer overflow panics
• Fixed a nil pointer panic when starting up when the resolver parent directory is missing (#8329)

MQTT

• Partial CONNECT packets can no longer exhaust pre-authentication memory
• PUBLISH remaining-length underflow no longer causes a server panic
• Subscriptions to internal $MQTT.deliver.pubrel subjects are now rejected
• Subscribe deny rules are now enforced on retained message and QoS replay paths
• WebSocket /mqtt upgrades no longer panic when MQTT is disabled

Monitoring

• JetStream remote usage updates no longer panic on length integer overflow

JetStream

• A data race on the cluster meta node during JetStream shutdown has been fixed (#8260)
• Meta proposal inflight tracking is now kept consistent during stream moves and related operations (#8261)
• Stream catchup is no longer skipped when limits are exceeded, preventing possible stream desync (#8265)
• Malformed TTL and schedule state is now rejected during decode (#8269)
• Zero consumer limits are now treated as unlimited during stream updates (#8286)
• Raft nodes no longer participate in voting or candidacy after write errors (#8290)
• Raft checkpoint handling now aborts if the node is closed (#8296)
• Raft ApplyCommit now handles the post-snapshot index correctly (#8321)
• Consumer ack subscriptions now match correctly when consumer names contain % (#8301)
• Atomic batch end-of-batch max-size checks and R1 message rewrites have been fixed (#8305)
• Peer state decoding now bounds peer ID reads to the buffer length (#8310)
• Counter stream staging no longer corrupts the committed running total (#8311)
• Filestore compaction no longer corrupts compressed or encrypted blocks (#8312)
• Memory store NumPending no longer overcounts for DeliverLastPerSubject consumers (#8313)
• Consumer inactive-delete grace period handling and pull request MaxBytes budgeting have been fixed (#8314)
• MultiLastSeqs no longer reorders stream config subjects through filterIsAll handling (#8315)
• Meta recovery snapshots no longer leave phantom streams or consumers behind (#8324)
• Skipped messages last time no longer violates ordering that could lead to issues with starting by time (#8237)
• Raft now reverts uncommitted membership changes correctly when truncating or snapshotting (#8332)

Credits

While CVE advisory notices are credited individually, a number of fixes in this release were the result of non-CVE reports from the following contributors:

• Koda Reef
• @Emin-ACIKGOZ
• @0xVijay
• Yaohui Wang
• @alanturing881

Complete Changes

https://github.com/nats-io/nats-server/compare/v2.12.11...v2.12.12

Changelog

Refer to the 2.12 Upgrade Guide for backwards compatibility notes with 2.11.x.

Go Version

• 1.25.11

Fixed

JetStream

• Fixed a regression introduced in v2.12.7 which could result in stale subject state tracking which could manifest in "Message Not Found" errors when a max messages per subject limit is configured (#8285)
  • Please note that v2.14.x versions are not affected

Complete Changes

https://github.com/nats-io/nats-server/compare/v2.12.10...v2.12.11

Changelog

Refer to the 2.14 Upgrade Guide for backwards compatibility notes with 2.12.x. Please note that the 2.13.x version was skipped.

Go Version

• 1.26.3

Dependencies

• golang.org/x/crypto v0.52.0
• golang.org/x/sys v0.45.0
• github.com/nats-io/jwt/v2 v2.8.2
• github.com/nats-io/nkeys v0.4.16

Improved

General

• The client ID is now available through the embedded ClientAuthentication API (#8217)

Fixed

General

• A race condition when handling subscription interest over routes has been fixed (#8235)
• Potential protocol-level corruption from rewriting $JS.ACK subjects has been fixed (#8242)
• Potential protocol-level corruption from buffer misuse in compressed WebSocket clients has been fixed (#8244)
• The /accstatz monitoring endpoint no longer omits accounts with only leaf connections (#8252)

JetStream

• Fixed a case where Raft peers were not correctly tracked after an inactivity stall during catchup (#8226)
• Quorum needed is now calculated correctly when bootstrapping the metalayer when gateway URLs resolve to multiple IP addresses (#8238)
• The filestore no longer performs a block skip check on streams with extremely high subject counts, as it could result in runaway CPU usage (#8227)
• Fixed a case where the filestore would not release a lock after handling a write error (#8232)
• Purge operations on both file and memory stores are now more consistent with each other (#8241)
• Fixed a case where the consumer lock would not release a lock after handling a start sequence error (#8230)
• Counter streams and message schedules now have configuration constraints applied to prevent incorrect usage patterns (#8240)
• Improved stream and consumer scale down behaviour consistency (#8253)
• Fixed an issue where the per-subject state last block was not stored correctly with a max messages per subject limit of 1 (#8254)
• Fixed a drift that could occur in the peer sets after a peer remove of an online node (#8258)

Complete Changes

https://github.com/nats-io/nats-server/compare/v2.14.1...v2.14.2

Changelog

Refer to the 2.12 Upgrade Guide for backwards compatibility notes with 2.11.x.

Go Version

• 1.25.10

Dependencies

• golang.org/x/crypto v0.52.0
• golang.org/x/sys v0.45.0
• github.com/nats-io/jwt/v2 v2.8.2
• github.com/nats-io/nkeys v0.4.16

Improved

General

• The client ID is now available through the embedded ClientAuthentication API (#8217)

Fixed

General

• A race condition when handling subscription interest over routes has been fixed (#8235)
• Potential protocol-level corruption from rewriting $JS.ACK subjects has been fixed (#8242)
• Potential protocol-level corruption from buffer misuse in compressed WebSocket clients has been fixed (#8244)
• The /accstatz monitoring endpoint no longer omits accounts with only leaf connections (#8252)

JetStream

• Fixed a case where Raft peers were not correctly tracked after an inactivity stall during catchup (#8226)
• Quorum needed is now calculated correctly when bootstrapping the metalayer when gateway URLs resolve to multiple IP addresses (#8238)
• The filestore no longer performs a block skip check on streams with extremely high subject counts, as it could result in runaway CPU usage (#8227)
• Purge operations on both file and memory stores are now more consistent with each other (#8241)
• Counter streams and message schedules now have configuration constraints applied to prevent incorrect usage patterns (#8240)
• Improved stream and consumer scale down behaviour consistency (#8253)
• Fixed an issue where the per-subject state last block was not stored correctly with a max messages per subject limit of 1 (#8254)
• Fixed a drift that could occur in the peer sets after a peer remove of an online node (#8258)

Complete Changes

https://github.com/nats-io/nats-server/compare/v2.12.9...v2.12.10

Changelog

Refer to the 2.14 Upgrade Guide for backwards compatibility notes with 2.12.x. Please note that the 2.13.x version was skipped.

Go Version

• 1.26.3

Dependencies

• golang.org/x/crypto v0.52.0
• golang.org/x/sys v0.45.0

Fixed

General

• A race condition when handling subscription interest over routes has been fixed (#8235)
• Potential protocol-level corruption from rewriting $JS.ACK subjects has been fixed (#8242)
• Potential protocol-level corruption from buffer misuse in compressed WebSocket clients has been fixed (#8244)

JetStream

• Fixed a case where Raft peers were not correctly tracked after an inactivity stall during catchup (#8226)
• Quorum needed is now calculated correctly when bootstrapping the metalayer when gateway URLs resolve to multiple IP addresses (#8238)
• The filestore no longer performs a block skip check on streams with extremely high subject counts, as it could result in runaway CPU usage (#8227)
• Fixed a case where the filestore would not release a lock after handling a write error (#8232)
• Purge operations on both file and memory stores are now more consistent with each other (#8241)
• Fixed a case where the consumer lock would not release a lock after handling a start sequence error (#8230)
• Counter streams and message schedules now have configuration constraints applied to prevent incorrect usage patterns (#8240)

Complete Changes

https://github.com/nats-io/nats-server/compare/v2.14.1...v2.14.2-RC.1

Changelog

Refer to the 2.12 Upgrade Guide for backwards compatibility notes with 2.11.x.

Go Version

• 1.25.10

Dependencies

• golang.org/x/crypto v0.52.0
• golang.org/x/sys v0.45.0

Fixed

General

• A race condition when handling subscription interest over routes has been fixed (#8235)
• Potential protocol-level corruption from rewriting $JS.ACK subjects has been fixed (#8242)
• Potential protocol-level corruption from buffer misuse in compressed WebSocket clients has been fixed (#8244)

JetStream

• Fixed a case where Raft peers were not correctly tracked after an inactivity stall during catchup (#8226)
• Quorum needed is now calculated correctly when bootstrapping the metalayer when gateway URLs resolve to multiple IP addresses (#8238)
• The filestore no longer performs a block skip check on streams with extremely high subject counts, as it could result in runaway CPU usage (#8227)
• Purge operations on both file and memory stores are now more consistent with each other (#8241)
• Counter streams and message schedules now have configuration constraints applied to prevent incorrect usage patterns (#8240)

Complete Changes

https://github.com/nats-io/nats-server/compare/v2.12.9...v2.12.10-RC.1

Changelog

Refer to the 2.14 Upgrade Guide for backwards compatibility notes with 2.12.x. Please note that the 2.13.x version was skipped.

Go Version

• 1.26.3 (#8107)

Dependencies

• github.com/klauspost/compress v1.18.6 (#8124)
• golang.org/x/crypto v0.51.0 (#8124)
• golang.org/x/sys v0.44.0 (#8124)

Added

General

• New metrics in_client_msgs, in_client_bytes, out_client_msgs and out_client_bytes are now available via the /varz monitoring endpoint for tracking data to/from normal clients only (#7851)

Improved

General

• Client TLS certificates without subject DNs but with DNS subject alternate names are now permitted (#8100)
• The log level of TLS handshake timeout or non-TLS record errors have been demoted to debug level to reduce noise (#8096)

JetStream

• Num pending is now only calculated on consumer leaders, avoiding unnecessary CPU usage on followers (#8172)
• Snapshot and catchup loops no longer leak timers (#8186, thanks to @SebTardif)
• Stream and consumer assignment errors are now surfaced (#8208)
• Intersection of sublists and subject trees can now be cancelled early, avoiding high CPU usage in some pathological cases (#8209)

Fixed

General

• Cluster route compression now obeys the cluster max_pings_out option if configured (#8093)
• The internal send loop no longer mutates caller headers, which could corrupt buffers (#8097)
• Removing headers no longer fails to remove later headers if the matching prefix also appeared in an earlier header value (#8103)
• The sublist now correctly maintains negative results in the cache when calculating number of interested subjects (#8119)
• Server shutdown requests are now idempotent, preventing concurrency issues when shutting down in embedded contexts (#8163)
• TLS listeners now work correctly with the PROXY protocol where enabled (#8130)
• Reduced lock contention that could be created between leafnodes and clients (#8139, #8159)
• Fixed a panic that could happen when an error occurs when walking JWT directory resolver folders (#8173, thanks to @SebTardif)
• In-process connections will no longer unexpectedly revert to TLS required with async INFO (#8205)

Leafnodes

• Leafnode connections will no longer negotiate compression if they are configured over already-compressed WebSockets (#7969)

JetStream

• Fast batch now correctly parses the batch sequence as a uint64 (#8094)
• Atomic batch no longer double-pools committed entries on cleanup (#8098)
• Raft nodes will now ignore temporary snapshots on recovery after a crash (#8101)
• A number of paths that could leave consumer redelivered in a drifted state have been fixed, e.g. with workqueue or interest-based streams with max_deliver, on single message removal or after purges/compactions (#8102)
• Caches are now cleared correctly when converting filestore encryption mode, avoiding block-level corruption (#8105, #8166)
• Fixed a race condition when updating the deduplication map on leader change (#8106)
• Source consumer creation will no longer schedule a recreation if a setup is already in progress, avoiding potential setup storms (#8111)
• Fixed data races when reading from the stream configuration when checking reservations, answering some API requests amongst others (#8115)
• Stream republish subjects are now validated correctly (#8127)
• The delivery policy for consumers on clustered workqueue streams is now enforced correctly (#8126)
• The Nats-Schedule-Next: purge action now correctly checks if the target is a schedule (#8135)
• Raft node append entry caches are now invalidated correctly on WAL truncation and snapshot installs (#8149)
• Skip message errors are now surfaced correctly, propagating failures (#8152)
• Mirror consumers are now retried immediately on a last sequence mismatch, avoiding stalling for longer than necessary (#8152)
• Raft nodes will no longer allow proposals to remove unknown peers (#8154)
• Pending state no longer leaks when reaching max deliveries (#8156)
• A panic when reusing a wait group when resetting a stream's clustered state has been fixed (#8158)
• Correctly reset local meta log when extending the meta group to a parent domain (#8142)
• Consumer file stores will now correctly flush when deleting a single redelivery, avoiding unexpected further redeliveries (#8168)
• Storage reservations for un-tiered streams have been made consistent between creates/updates and clustered/non-clustered modes (#8170)
• Raft will now correctly cancel an in-flight checkpoint operation when resetting (#8180, #8202)
• The JetStreamMaxMemory and JetStreamMaxStore options are now handled correctly in embedded mode (#8184)
• A number of fields that were aliasing underlying filestore block caches have been fixed (#8187)
• Consumers with inactive_threshold should no longer have their local state deleted unexpectedly when the proposal to the metalayer to clean up the consumer fails (#8198)
• Metalayer state is now preserved in a number of cases where it was incorrectly being removed on shutdown (#8199)

MQTT

• Invalid characters in subjects are now rejected correctly, avoiding protocol issues when forwarded to other connection types (#8104, #8112)

Complete Changes

https://github.com/nats-io/nats-server/compare/v2.14.0...v2.14.1