Netdata

Release notes

Netdata v2.10.4 is a patch release to address issues discovered since v2.10.3.

This is a larger-than-usual patch release focused on stability, memory safety, and crash resilience across the database engine, streaming, Cloud connectivity, and collectors. It also backports the new macOS hardware sensor collectors to the 2.10.x line.

New in this release

• macOS hardware sensor collectors: GPU power, clock, and temperatures, SMC and IOHID sensors and fans, power sources and thermal pressure, NVMe SMART, with a powermetrics fallback via ndsudo, plus a cross-OS sensors function and a shared temperature-histogram context (#22475, #23085, @ktsaou)
• Bounded-cardinality process grouping for apps.plugin on macOS, keeping per-process monitoring meaningful on hosts with high process churn (#23085, @ktsaou)

Database engine and metadata

• Hardened DBENGINE against corrupted or truncated data files: extent disk-size and uncompressed-page-bounds validation, SIGBUS protection and header-offset bounds checks on v2 journal walks, improved journal file access error handling, and spinlock-holder identification for datafile/journal deadlock diagnostics (#22324, #22514, #22310, #22725, @stelfrag; #22666, @jmestwa-coder)
• Fixed page-cache races in pgc_page_add and pgc_queue_del, a deadlock on dimension creation, and a crash on virtual node takeover (#22466, #22400, #22417, @stelfrag)
• Validated data loaded from SQLite to prevent crashes on corrupted databases, and improved UUID handling and error reporting in SQLite functions (#22679, #22233, @stelfrag)
• Kept chart indexes allocated while a host is archived and guarded against a null root index in chart lookups, avoiding null dereferences during queries (#23074, #23056, @stelfrag)
• Fixed an rrdcontext metadata leak on non-dbengine hosts (#22438, @stelfrag)
• Rotated the active datafile and requeued the pending extent on unrecoverable write errors, so a failing disk no longer stalls the database engine (#23048, @ktsaou)

Streaming and replication

• Tracked and accounted memory allocation size in replication queries, and fixed a sender replication counter leak on obsolete charts (#22756, #22428, @stelfrag)
• Rejected oversized ZSTD frames and handled decompression errors gracefully (log and fail the connection instead of fatal()) (#22830, @stelfrag)
• Fixed the streaming receiver discarding already-delivered data when a child disconnects (#23118, @ktsaou)

Cloud connectivity (ACLK)

• Prevented rare unbounded one-core CPU spins in the cloud-connection loops, and avoided returning an uninitialized packet_id on publish failure (#22879, @ktsaou; #22504, @stelfrag)

Alerts and machine learning

• Bounded the alert notification execution wait so a stuck notification script cannot block progress, and fixed a shutdown race when restoring alert information from the database (#22626, #22448, @stelfrag)
• Added safeguards against ML database corruption and streamlined the recovery process (#22478, @stelfrag)

Collectors

• Fixed systemd-journal.plugin memory retention after queries and its apps.plugin accounting (#23089, @ktsaou)
• Fixed file descriptor accounting in apps.plugin (adding nfds monitoring) and eBPF FD PID map iteration (#22447, @arch-yunus; #22436, @stelfrag)
• Fixed buffer overflows and counter-size issues in platform collectors: a stack buffer overflow in macOS mach_smi, FreeBSD counter size mismatches and an off-by-one in freebsd_ipfw and the claim code, and a freeipmi.plugin watchdog underflow at low system uptime (#22553, @artem; #23044, @DavidMarec; #22710, @vkalintiris; #22490, @ktsaou)
• Fixed Windows Events row rendering crashes with stricter size checks, safer XML parsing, and robust variant handling (#22872, @ktsaou)
• Restored stable temperature and power collection in go.d/nvidia_smi with NVIDIA driver 580 XML output variants (#23047, @copilot-swe-agent)
• Fixed Windows hardware detection so virtual machines are no longer reported as bare metal (#22942, @thiagoftsm)
• Moved the fail2ban socket path into ndsudo for go.d/fail2ban (#22745, @ilyam8)
• Fixed a pluginsd cleanup race with an active collector, added a slot bounds check to the pluginsd parser, and initialized mountpoint state before the slow worker in the diskspace plugin (#22207, #22598, #22298, @stelfrag)

Security and API

• The /api/v3/settings endpoint is now gated behind HTTP_ACL_DASHBOARD instead of HTTP_ACL_NOCHECK, enforcing connection allowlists and blocking anonymous state-changing requests (#22896, @stelfrag)
• Fixed the csvjsonarray output format emitting invalid JSON when label-quotes is passed (#23115, @ktsaou)

Core stability and memory safety

• Fixed internal race conditions: a TOCTOU race in dictionary_destroy(), ARAL allocator races and a double-free (#22108, #22212, #22294, #22367, @stelfrag)
• Fixed a payload memory leak, JSON object validation and null-key handling, an inverted empty-buffer guard in simple_pattern, and prevented a concurrent fatal from masking the first one (#22492, #22309, #22326, #22671, @stelfrag; #22688, @vkalintiris)
• Fixed memory-safety and correctness bugs surfaced by Coverity and Sonar audits (#22266, #22267, #22268, #22270, #22279, #22281, #22329, #22330, #22331, #22332, #22333, #22334, #22335, #22336, #22338, #22339, #22340, #22835, @stelfrag; #22293, @ktsaou)
• Improved shutdown robustness: fixed a SIGSEGV in ctx_hosts_load and pipe cleanup handling of the ENOENT case (#22419, #22509, @stelfrag)

Installation and packaging

• Windows: fixed MSI installer issues, adjusted the installer to run claiming only when both the token and rooms are provided (with safer wide-character handling and a capped HTTP claim response size), and fixed the search for Visual Studio tooling to check for multiple versions (#22751, #22754, @thiagoftsm; #22723, @Ferroin)
• Improved error handling in netdata-updater when fetching files and silenced curl output (#22422, @Ferroin; #22639, @AJCxZ0)

Support options

As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:

• Netdata Learn: Find documentation, guides, and reference material for monitoring and troubleshooting your systems with Netdata.
• GitHub Issues: Make use of the Netdata repository to report bugs or open a new feature request.
• GitHub Discussions: Join the conversation around the Netdata development process and be a part of it.
• Community Forums: Visit the Community Forums and contribute to the collaborative knowledge base.
• Discord Server: Jump into the Netdata Discord and hang out with like-minded sysadmins, DevOps, SREs, and other troubleshooters. More than 2000 engineers are already using it!

Release notes

Netdata v2.10.3 is a patch release to address issues discovered since v2.10.2.

This patch release provides the following bug fixes and updates:

• Fixed a per-PID shared-memory pool leak in ebpf.plugin that, on hosts with normal process churn, filled the 32,768-slot pool within ~15 hours and then pegged a CPU core at 100% in an infinite map-iteration loop; aggregation paths now use a non-allocating lookup, freshly-allocated slots are zeroed, module bits are swept on exit, and stale shared-memory and semaphore objects are properly cleaned up on init (#22232, @ktsaou)
• Switched the SNMP collector's primary uptime source to SNMP-FRAMEWORK-MIB::snmpEngineTime (in seconds) to avoid the ~497-day TimeTicks wrap of hrSystemUptime/sysUpTime, while keeping the existing systemUptime metric name and HR-MIB fallback (#22231, @ilyam8)
• Split dynamic configuration job-name validation per domain so service discovery, vnode, and secret-store names can include dots (e.g., FQDNs), while collectors retain strict naming rules (#22247, @ilyam8)
• Removed the unused extra_details field from the go.d/powerstore Hardware struct to fix /hardware response decoding errors that caused job check failures (#22291, @ilyam8)

Support options

As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:

• Netdata Learn: Find documentation, guides, and reference material for monitoring and troubleshooting your systems with Netdata.
• GitHub Issues: Make use of the Netdata repository to report bugs or open a new feature request.
• GitHub Discussions: Join the conversation around the Netdata development process and be a part of it.
• Community Forums: Visit the Community Forums and contribute to the collaborative knowledge base.
• Discord Server: Jump into the Netdata Discord and hang out with like-minded sysadmins, DevOps, SREs, and other troubleshooters. More than 2000 engineers are already using it!

Release notes

Netdata v2.10.2 is a patch release to address issues discovered since v2.10.1.

This patch release provides the following bug fixes and updates:

• Fixed ZFS-related crashes in diskspace.plugin by guarding a NULL filesystem field and replacing the blocking pool-capacity collector with a lightweight cache fed by existing statvfs calls, preventing coredumps on degraded or exporting ZFS pools (#22188, @thiagoftsm)
• Reduced default SNMP MaxOIDs from 60 to 20 for more reliable polling, and removed 32-bit counter fallbacks from the IF-MIB profile to prevent counter type switching and overflow between collection cycles (#22203, @ilyam8)
• Eliminated false-positive "timed out waiting for enable/disable decision" warnings by removing the wait-decision timeout and making the dyncfg command path non-droppable, so back-pressure flows upstream instead of producing 503 errors (#22201, @ilyam8)

Support options

As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:

• Netdata Learn: Find documentation, guides, and reference material for monitoring and troubleshooting your systems with Netdata.
• GitHub Issues: Make use of the Netdata repository to report bugs or open a new feature request.
• GitHub Discussions: Join the conversation around the Netdata development process and be a part of it.
• Community Forums: Visit the Community Forums and contribute to the collaborative knowledge base.
• Discord Server: Jump into the Netdata Discord and hang out with like-minded sysadmins, DevOps, SREs, and other troubleshooters. More than 2000 engineers are already using it!

Release notes

Netdata v2.10.1 is a patch release to address issues discovered since v2.10.0.

This patch release provides the following bug fixes and updates:

• Added ping_only option to the SNMP collector, allowing collection of only ICMP round-trip time metrics while still using SNMP at startup for device identification and labeling (#22180, @ilyam8)
• Fixed SNMP collector retries serialization so a zero value is correctly preserved, and disabled bulk walk detection when MaxRepetitions is set to 0 (#22179, @ilyam8)
• Reduced transient 503 errors on dynamic configuration enable/disable by buffering the command channel and extending the handoff timeout in the go.d plugin (#22183, @ilyam8)

Support options

As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:

• Netdata Learn: Find documentation, guides, and reference material for monitoring and troubleshooting your systems with Netdata.
• GitHub Issues: Make use of the Netdata repository to report bugs or open a new feature request.
• GitHub Discussions: Join the conversation around the Netdata development process and be a part of it.
• Community Forums: Visit the Community Forums and contribute to the collaborative knowledge base.
• Discord Server: Jump into the Netdata Discord and hang out with like-minded sysadmins, DevOps, SREs, and other troubleshooters. More than 2000 engineers are already using it!

Table of Contents

• Summary
• Highlights
• Acknowledgments
• Contributions
  • Collectors
  • Packaging/Installation
  • Documentation
  • Other Notable Changes
• Deprecation notice
• Support options

Release Summary

Netdata v2.10.0 introduces secrets management, a Nagios plugins collector, Azure Monitor support, and broad stability improvements.

Feature	Highlights	Details
Secrets Management	4 resolver types, 4 backends	• Environment variables, files, commands, and secretstore references• AWS Secrets Manager, Azure Key Vault, GCP Secret Manager, HashiCorp Vault
Enhanced AI Conversations	Reports and Alerts integration	• Resume and expand generated AI reports • Ask AI about active alerts or charts, their root causes, and recommended remediation steps
Advanced Alerting	Silencing, Evaluation & Acknowledge	• Complete support for recurrence rules for Alerts Silencing • Test alert definitions against historical data before deploying • Acknowledge alerts you're working on
Improved Custom Dashboards	Complete Customisation	• Add Metrics, Logs, Events, Node List, Alerts, Live Functions and Text components • Duplicate Custom Dashboards
Nagios Plugins Collector	Run any Nagios-compatible check	• Automatic performance data charts• Threshold-based alerting with soft/hard state logic• Execution metrics (duration, CPU, memory)
Azure Monitor Collector	38 service profiles, 1,300+ metrics	• Automatic resource discovery via Azure Resource Graph• Multi-subscription, flexible scoping by resource groups/regions/tags
Azure AD for Database Collectors	MSSQL, PostgreSQL, Generic SQL	• Service principal, managed identity, and default credential chain• Passwordless auth for Azure-hosted databases
New Storage Collectors	Dell PowerStore, Dell PowerVault	• Hardware health, performance, capacity, and sensor monitoring• Built on V2 collector framework
Expanded Collector Coverage	vSphere, MSSQL, SNMP, Docker	• vSphere datastores/clusters/resource pools• MSSQL Always On AG monitoring• SNMP IPSec/VPN profiles for FortiGate, Juniper, MikroTik, Check Point• Docker container listing function
OpenTelemetry Improvements	Metrics pipeline overhaul	• Proper slot-based aggregation with configurable intervals• Multi-slot ingestion with out-of-order support
Stability & Performance	Production reliability	• Faster agent startup• ML prediction optimization• Alerts API speedup• Multiple crash and race condition fixes

Release Highlights

Secrets Management

Keep collector credentials out of plain-text configuration files. Netdata now lets you reference secrets in collector configurations instead of storing them directly. Passwords, tokens, and API keys are resolved at runtime from the source you choose.

Resolvers

Netdata provides four resolver types, from simple to enterprise-grade:

Resolver	Syntax	Best For
Environment variable	${env:VAR_NAME}	Secrets already injected into the Netdata service environment
File	${file:/absolute/path}	Secrets stored in local files on disk (Docker Secrets, Kubernetes Secrets mounted as volumes)
Command	${cmd:/absolute/path args}	Secrets returned by a trusted local command (e.g., 1Password CLI, custom scripts)
Secretstore	${store:<kind>:<name>:<operand>}	Secrets managed centrally in cloud providers or Vault

Supported Secretstore Backends

For organizations managing secrets centrally, Netdata integrates with four secretstore backends:

Backend	Kind	Operand Format	Authentication Modes
AWS Secrets Manager	aws-sm	secret-name[#key]	Environment credentials, ECS task role, EC2 instance profile
Azure Key Vault	azure-kv	vault-name/secret-name	Service principal, Managed identity, Default credential chain
Google Secret Manager	gcp-sm	project/secret[/version]	Metadata server, Service account file
HashiCorp Vault	vault	path#key	Token, Token file

Configure secretstores through the Netdata Dynamic Configuration UI (Collectors → go.d → SecretStores) or via configuration files under /etc/netdata/go.d/ss/.

Example

A MySQL collector referencing its password from HashiCorp Vault:

# /etc/netdata/go.d/mysql.conf
jobs:
  - name: mysql_prod
    dsn: "netdata:${store:vault:vault_prod:secret/data/netdata/mysql#password}@tcp(127.0.0.1:3306)/"

You can mix resolver types freely. For example, read the username from an environment variable and the password from a secretstore in the same config value:

jobs:
  - name: mysql_prod
    dsn: "${env:MYSQL_USER}:${store:vault:vault_prod:secret/data/netdata/mysql#password}@tcp(127.0.0.1:3306)/"

How It Works

• Secrets are resolved each time a collector job starts or restarts.
• Updating a secretstore automatically restarts collector jobs that use it so they pick up new credentials.
• Secretstore configuration values themselves also support ${env:...}, ${file:...}, and ${cmd:...} resolvers, so backend credentials don't need to be stored in plain text either.

[!TIP] For detailed setup, backend-specific authentication guides, and troubleshooting, see the Secrets Management documentation.

Enhanced AI Conversations

Netdata AI Conversations are now more powerful. You can now add Active Alerts, Charts or AI-generated reports to start an AI conversation to provide better, more meaningful responses to your conversations.

Advanced Alerting

With this release, Netdata provides our users with the ability to:

• Graphically Evaluate Alerts on historical data before they are deployed

• Create complex recurrence rules on Alerts Silencing to manage your maintenance windows

• Acknowledge Alerts that your teams are working on

Improved Custom Dashboards

With this release, we have improved our custom dashboards support to:

• Add metrics charts, logs, live functions, events feeds, active alerts counts, node list, etc.
• Duplicate Dashboards to help create new custom dashboards from existing ones

Nagios Plugins Collector

Run Nagios-compatible plugins and custom scripts directly inside Netdata with the new Nagios Plugins collector. Any executable that follows the Nagios plugin output format works out of the box: packaged Nagios plugins, custom shell scripts, or compiled binaries.

What You Get

Feature	Details
Check State Monitoring	Tracks OK, WARNING, CRITICAL, and UNKNOWN states from the process exit code
Automatic Performance Data Charts	Nagios performance data values are parsed and charted automatically
Threshold-Based Alerting	When perfdata includes warning/critical thresholds, Netdata derives a threshold state chart (ok/warning/critical) and creates built-in alerts
Execution Metrics	Measures run duration, CPU time, and memory usage of each check
Nagios-Style Scheduling	Configurable check intervals, retry intervals, max check attempts, and time periods
Soft/Hard State Logic	Non-OK results retry at retry_interval up to max_check_attempts before triggering alerts, preventing false alarms from transient failures

Built-in Alerts

The collector ships with four stock alerts:

Alert	Triggers When
nagios_job_execution_state_warn	A check is in WARNING hard state
nagios_job_execution_state_crit	A check is in CRITICAL hard state
nagios_job_perfdata_threshold_state_warn	A perfdata metric exceeds its warning threshold
nagios_job_perfdata_threshold_state_crit	A perfdata metric exceeds its critical threshold

All stock alerts suppress soft retry states. Alerts fire only after the configured max_check_attempts consecutive failures.

Example

# /usr/local/lib/netdata/checks/check_api.sh
#!/bin/sh
resp=$(curl -s -o /dev/null -w "%{http_code} %{time_total}" --max-time 5 "$1" 2>/dev/null)
code=$(echo "$resp" | cut -d' ' -f1)
time=$(echo "$resp" | cut -d' ' -f2)

[ "$code" -ge 500 ] && echo "CRITICAL - HTTP $code | response_time=${time}s;~:2;~:5" && exit 2
[ "$code" -ne 200 ] && echo "WARNING - HTTP $code | response_time=${time}s;~:2;~:5" && exit 1
echo "OK - HTTP $code | response_time=${time}s;~:2;~:5"
exit 0

# /etc/netdata/scripts.d/nagios.conf
jobs:
  - name: api_health
    plugin: /bin/sh
    args: ["/usr/local/lib/netdata/checks/check_api.sh", "http://localhost:8080/health"]
    check_interval: 1m
    retry_interval: 15s
    max_check_attempts: 3

This produces check state charts, execution metrics, and because the script emits perfdata with thresholds (response_time with warn >2s / crit >5s), an automatic response time chart with a threshold state chart and built-in alerts.

[!TIP] For the full setup guide, custom scripts walkthrough, and time period configuration, see the Nagios Plugins collector documentation.

Azure Monitor Collector

Monitor your entire Azure infrastructure from Netdata with the new Azure Monitor collector. It collects platform metrics from the Azure Monitor Metrics API with automatic resource discovery: point it at your subscriptions, and it finds your resources, matches them to built-in profiles (preconfigured sets of metrics), and starts collecting metrics.

[!IMPORTANT] This is an alpha feature. The configuration format may change in upcoming releases.

What You Get

Feature	Details
38 Built-in Service Profiles	1,300+ metrics across compute, databases, networking, storage, AI/ML, analytics, containers, and more
Automatic Discovery	Discovers resources via Azure Resource Graph and enables matching profiles without per-resource configuration
Multi-Subscription	Monitor resources across multiple Azure subscriptions in a single collector job
Flexible Discovery Scoping	Narrow discovery by resource groups, regions, or tags, or use a custom Azure Resource Graph KQL query
Built-in Alerts	Pre-configured alerts for critical Azure resource conditions
UI Configuration	Configure through the Netdata Dynamic Configuration UI without file editing

Supported Services

Category	Services via Profiles
Compute	Virtual Machine, Virtual Machine Scale Set, App Service (incl. Functions), Container Apps, Container Instances
Kubernetes	AKS
Databases	SQL Database, SQL Elastic Pool, SQL Managed Instance, MySQL Flexible, PostgreSQL Flexible, Cosmos DB, Cache for Redis
Networking	Load Balancer, Application Gateway, Front Door, Firewall, NAT Gateway, VPN Gateway, ExpressRoute Circuit, ExpressRoute Gateway
Storage & Analytics	Storage Account, Data Explorer, Data Factory, Stream Analytics, Synapse Analytics, Log Analytics
Messaging	Event Hubs, Service Bus, Event Grid, IoT Hub
AI & ML	Cognitive Services, Machine Learning, Application Insights
Integration	API Management, Logic Apps, Key Vault, Container Registry

[!TIP] For detailed setup instructions, authentication options, custom profiles, and tuning, see the Azure Monitor collector setup guide.

Acknowledgments

• @kiwixz for increasing statsd UDP buffer size to match localhost MTU, preventing packet truncation.
• @DarkByteZero for fixing MongoDB top-queries timeout, netdev_mutex deadlock, and apps.plugin use-after-free crash.

Contributions

Collectors

• Overhauled OpenTelemetry metrics pipeline with proper slot-based aggregation, multi-slot ingestion with out-of-order support, configurable intervals, and optimized JSON handling (otel.plugin). (#21771, #21893, #21896, #22085, @vkalintiris)
• Added secrets management support, allowing collector credentials to be stored in external secret stores like AWS Secrets Manager, Azure Key Vault, GCP Secret Manager, and HashiCorp Vault (go.d). (#21951, #22081, #22083, @ilyam8)
• Rewrote scripts.d collector to V2 framework with alertable state support for Nagios-style custom scripts (go.d/scripts.d). (#21908, #22008, @ilyam8)
• Added Azure Monitor collector for monitoring Azure cloud resources with built-in alerts (go.d/azure_monitor). (#21993, #22007, #22095, @ilyam8)
• Added Azure AD authentication support for MSSQL, PostgreSQL, and generic SQL collectors (go.d). (#21905, #21995, @ktsaou, @ilyam8)
• Added Dell PowerVault ME4/ME5 storage array collector with system health, hardware status, performance, and capacity monitoring (go.d/powervault). (#21936, @ktsaou)
• Added Dell PowerStore storage array collector with cluster, appliance, volume, node, and hardware monitoring (go.d/powerstore). (#21929, @ktsaou)
• Added Always On Availability Group monitoring to MSSQL collector with AG health, replica states, sync metrics, and WSFC cluster status (go.d/mssql). (#21927, @ktsaou)
• Added IPSec/VPN monitoring profiles for FortiGate, Juniper, MikroTik, and Check Point (go.d/snmp). (#21926, @ktsaou)
• Added per-Windows-service process tree grouping to apps.plugin with auto-discovery via Service Control Manager (apps.plugin). (#21925, @ktsaou)
• Added datastore, cluster, and resource pool monitoring to vSphere collector with capacity, IOPS, and latency metrics (go.d/vsphere). (#21924, @ktsaou)
• Added Docker container listi1ng function for interactive container overview in the dashboard (go.d/docker). (#21868, @ilyam8)
• Increased statsd UDP buffer size to match localhost MTU, preventing packet truncation (statsd). (#21822, @kiwixz)
• Improved Windows plugin with CPU temperature chart visibility and Control Panel enhancements (windows.plugin). (#21797, @thiagoftsm)

• Fixed dynamic configuration rollback for non-disruptive Service Discovery update failures (go.d). (#21861, @ilyam8)
• Fixed Service Discovery pipeline name derivation from source context (go.d). (#22105, @ilyam8)
• Fixed collector static functions not registering on first job start (go.d). (#21894, @ilyam8)
• Fixed MongoDB top-queries function timeout due to incorrect default timeout (go.d/mongodb). (#21860, @DarkByteZero)
• Fixed smartctl collector treating non-fatal exit codes as errors when disk health data is still valid (go.d/smartctl). (#21858, @ilyam8)
• Fixed netdev_mutex deadlock when /proc/net/dev reading fails, permanently blocking network interfaces (proc.plugin). (#21839, @DarkByteZero)
• Fixed apps.plugin use-after-free crash on parent pointer dereference during process collection (apps.plugin). (#21838, @DarkByteZero)
• Fixed service discovery to skip unsupported discoverer configurations instead of failing (go.d). (#21818, @ilyam8)
• Fixed DCGM exporter service discovery (go.d). (#21800, @ilyam8)

• Refactored Azure Monitor discovery and profile selection with proper configuration modes (go.d/azure_monitor). (#22088, #22095, @ilyam8)
• Improved Windows plugin codebase with internal improvements and fixes (windows.plugin). (#22039, @thiagoftsm)
• Introduced V2 metrics collection framework with float dimension support, structured family handling, and redesigned function management (go.d). (#21769, #21825, #21850, #21851, #21859, #21909, #21979, @ilyam8)
• Changed "no instances configured" function response code to 422 (go.d). (#21903, @ilyam8)
• Removed legacy MSSQL collector from Windows plugin, completing migration to go.d (windows.plugin). (#21876, #21886, @thiagoftsm)
• Restructured go.d agent internals with improved dyncfg lifecycle, logger attribution, and codebase organization (go.d). (#21803, #21808, #21817, #21821, #21830, #21833, #21840, #21842, @ilyam8)
• Added conditional and rate-limited logging to go.d logger (go.d). (#21813, @ilyam8)

Packaging/Installation

• Optimized release build profile for runtime speed with fat LTO and opt-level=3 (build). (#22046, @vkalintiris)
• Fixed Windows config editor default in installer (packaging). (#21957, @stelfrag)
• Fixed systemd-journal.plugin permissions on offline installs (packaging). (#21953, @ralphm)
• Added Fedora 44 to CI and package builds (CI). (#21943, @Ferroin)
• Fixed control files for DEB packages causing uninstallation issues (packaging). (#21940, @Ferroin)
• Added Ubuntu 26.04 to CI and package builds (CI). (#21939, @Ferroin)
• Fixed Windows installer driver installation across different Windows versions (packaging). (#21911, @thiagoftsm)
• Updated Go toolchain to v1.26.0 (build). (#21866, @ilyam8)
• Dropped Ubuntu 20.04 from CI and package builds after upstream EOL (CI). (#21647, @Ferroin)
• Increased minimum language standards to C17 and C++17 with updated Protobuf and Abseil (build). (#21574, @Ferroin)
• Updated libbpf and synced eBPF repositories (build). (#21982, @thiagoftsm)

Documentation

• Added and improved Secrets Management documentation with secret store backend guides (#22028, #22030, #22099, #22106, @ilyam8)
• Added Windows Authentication setup guide for MSSQL collector (#22089, @ilyam8)
• Added section on changing the default port (#22084, @nedi-app)
• Added NVIDIA/Cumulus/Mellanox to SNMP collector vendor list (#22063, @nedi-app)
• Corrected SEND_EMAIL default value from YES to AUTO (#22057, @nedi-app)
• Clarified streaming protocol architecture as non-HTTP/HTTPS (#22055, @nedi-app)
• Rewrote chart template README with user-oriented content (#22049, @ilyam8)
• Added Installation section to metrics-centralization-points README (#22036, @nedi-app)
• Added Space Settings navigation guidance to remove-node.md (#22035, @nedi-app)
• Reordered collector README sections for better onboarding flow (#22033, @ilyam8)
• Fixed integration name collision (#22031, @ilyam8)
• Updated Windows installer and plugin documentation (#21888, #22021, @thiagoftsm)
• Improved Nagios collector documentation and metadata (#22014, #22016, #22020, @ilyam8)
• Added file permissions and ownership documentation for claim.conf (#22017, @nedi-app)
• Added deprecation notice for /api/v1/charts endpoint (#21974, @nedi-app)
• Added cloud.d directory contents section to claim README (#21966, @nedi-app)
• Added multi-device SNMPv3 configuration example with YAML anchors (#21955, @ilyam8)
• Updated kickstart URL to canonical get.netdata.cloud (#21938, @Ancairon)
• Improved Alerts Automation docs discoverability (#21932, @shyamvalsan)
• Linked setup option names to detailed configuration sections in integration docs (#21897, @ilyam8)
• Clarified retention in Parent with tiered storage examples (#21887, @ralphm)
• Renamed Top tab to Live tab across all documentation (#21872, #21873, #21875, @shyamvalsan, @ilyam8)
• Added query metrics skill documentation with troubleshooting FAQ (#21834, #21836, #21837, @ktsaou)
• Added User Statistics prerequisite for MySQL per-user metrics (#21814, @ilyam8)
• Added TV Mode documentation section (#21812, @Ancairon)
• Updated consul keywords for search discoverability (#21787, @Ancairon)
• Clarified default storage retention limits in documentation (#21780, @ktsaou)

Other Notable Changes

• Optimized ML prediction performance with circular buffer, improved model loading, and safer cluster center handling (ML). (#21795, #22042, #22073, #22104, @stelfrag)
• Sped up alerts API filtering with host status snapshots for efficient prefiltering (health). (#21984, @stelfrag)
• Improved streaming stability by reducing shutdown timing issues (streaming). (#21992, @thiagoftsm)
• Improved netdatacli ping to properly report agent readiness with distinct exit codes (0=ready, 1=initializing, 255=unreachable) (daemon). (#21965, @stelfrag)
• Added periodic timezone refresh for correct DST handling across all components (daemon). (#21944, @stelfrag)
• Improved logger by removing spinlock during I/O to prevent deadlocks (daemon). (#21928, @stelfrag)
• Improved agent startup and restart time with optimized mmap settings and prefetching (dbengine). (#21891, @stelfrag)
• Added rejection of incoming streaming connections for locally collected vnodes (streaming). (#21889, @stelfrag)
• Enforced Pushover API field length limits to prevent notification failures on long URLs (health/notifications). (#21882, @Copilot)
• Added SOCKS5 and SOCKS5H proxy support to ACLK for environments requiring SOCKS proxies (ACLK). (#21831, @stelfrag)
• Added environment variable expansion in host labels with ${VAR} and ${VAR:-default} syntax (daemon). (#21796, @ktsaou)
• Added SNI support for streaming SSL/TLS connections, enabling reverse proxy compatibility (streaming). (#21715, @Copilot)
• Preserved UTF-8 characters in RRD string fields instead of replacing them with underscores (daemon). (#21694, @ktsaou)
• Added YAML support to libnetdata and migrated log2journal to shared YAML parser (libnetdata). (#20544, @ktsaou)

• Fixed eBPF PID cleanup to avoid use-after-free during iteration (ebpf.plugin). (#22098, @stelfrag)
• Fixed undefined behavior in timezone TZif 64-bit parsing (daemon). (#22097, @stelfrag)
• Fixed ACLK SIGABRT during shutdown that could lead to database corruption (ACLK). (#22051, @thiagoftsm)
• Fixed cache SIGSEGV in aral during journal v2 processing (dbengine). (#22026, @thiagoftsm)
• Fixed SIGSEGV in context dimension entry lookup (dbengine). (#22025, @thiagoftsm)
• Fixed clock handling in eBPF and other plugins to prevent crashes (ebpf.plugin). (#22009, @thiagoftsm)
• Fixed lock order during context processing to avoid deadlock (dbengine). (#21996, @stelfrag)
• Fixed uninitialized vnode stale timeout leaking between host definitions (streaming). (#21983, @stelfrag)
• Fixed thread safety and initialization handling in Windows hardware info collection (windows.plugin). (#21885, #21958, @stelfrag)
• Fixed health API to properly acquire and release RRD instances (health). (#21952, @stelfrag)
• Fixed data race in ML training during host stop (ML). (#21844, @stelfrag)
• Fixed context hub cleanup with queue bounding and stale entry handling (dbengine). (#21832, @stelfrag)
• Fixed potential use-after-free in RAM mode metric release (dbengine). (#21809, @stelfrag)
• Fixed URL validation in cloud config that incorrectly rejected valid URLs (ACLK). (#21805, @stelfrag)
• Fixed crash when processing corrupted journal files (dbengine). (#21794, @stelfrag)
• Fixed cache page queue race condition with revalidation under lock (dbengine). (#21793, @stelfrag)
• Fixed FreeBSD 15.0 build failure due to ipfw structure changes (daemon). (#21843, @Copilot)
• Fixed race condition during pluginsd dimension array operations (plugins.d). (#21628, @stelfrag)

• Removed compilation warnings across Linux, Windows, BSD platforms and removed variable-length arrays for C11+ compatibility (build). (#21827, #21961, #22024, #22066, #22091, #22103, @thiagoftsm, @stelfrag)
• Improved DBENGINE log messages with structured identifiers, tier prefixes, and correct file names (dbengine). (#22045, #22047, #22053, #22086, @stelfrag)
• Optimized eBPF memory handling with inline Judy storage and reduced fragmentation (ebpf.plugin). (#22050, @stelfrag)
• Added ML unit tests for anomaly detection features (ML). (#22043, @stelfrag)
• Added dictionary benchmark tool (libnetdata). (#22041, @stelfrag)
• Created journal directory if missing before inotify watch setup (otel.plugin). (#22019, @vkalintiris)
• Fixed various Coverity issues across timezone, ML, and health modules (various). (#21777, #21878, #21971, #21986, #22054, #22094, @thiagoftsm, @stelfrag)
• Changed missing user configuration directory log to info level (daemon). (#21969, @vkalintiris)
• Refactored stream connection definition parsing for reusability (streaming). (#21921, @stelfrag)
• Optimized UID/GID cache updates in apps plugin to once per cycle (apps.plugin). (#21864, @stelfrag)
• Added input validation for socket connection definitions (libnetdata). (#21881, @stelfrag)
• Reduced log noise for journal indexing limit warnings on online files (journal-viewer). (#21816, @vkalintiris)
• Improved ACLK proxy logging with protocol details and credential status (ACLK). (#21789, @stelfrag)
• Fixed potential crash in expression evaluation print function (libnetdata). (#21790, @stelfrag)
• Improved datafile deletion with retry mechanism and synchronization (dbengine). (#21781, @stelfrag)
• Adjusted eBPF user ring buffer handling (ebpf.plugin). (#21676, @thiagoftsm)
• Improved metadata storage with prepared statements for chart and host labels (dbengine). (#21492, @stelfrag)
• Added float dimension support to plugins.d protocol and go.d framework (plugins.d, go.d). (#21349, #21825, @ktsaou, @ilyam8)

Deprecation notice

Changed in this release

Important Changes in Next Minor Release

Important Changes in Next Major Release

Deprecated Components

Component Type	Versions Being Deprecated
APIs	v1, v2

What This Means

Only the v3 API and v3 Dashboard will be supported starting with the next major release. These newer versions offer improved performance, enhanced features, and better security.

Support options

As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:

• Premium Support: Customers who wish to have a direct channel with Netdata and prioritized support with defined SLAs can contact us.
• Netdata Learn: Find documentation, guides, and reference material for monitoring and troubleshooting your systems with Netdata.
• GitHub Issues: Use the Netdata repository to report bugs or open a new feature request.
• GitHub Discussions: Join the conversation around the Netdata development process and be a part of it.
• Community Forums: Visit the Community Forums and contribute to the collaborative knowledge base.
• Discord Server: Jump into the Netdata Discord and hang out with like-minded sysadmins, DevOps, SREs, and other troubleshooters. More than 2000 engineers are already using it!

Table of Contents

• Summary
• Highlights
• Contributions
  • Collectors
  • Packaging/Installation
  • Documentation
  • Other Notable Changes
• Deprecation notice
• Support options

Release Summary

Netdata v2.9.0 brings powerful database observability and expanded OpenTelemetry support.

Feature	Highlights	Details
Interactive Query Analysis	14+ databases supported	• Identify slow queries• Debug bottlenecks• Monitor operations• No manual database connections required
Rebuilt SQL Server Collector	Rewritten in Go	• Comprehensive metrics• Query Store integration• UI-based configuration
OpenTelemetry Log Ingestion	Logs via otel plugin	• Stored in systemd-compatible journal files• Configurable retention policies
Stability Improvements	Production reliability	• Under-the-hood fixes for more robust operation

Release Highlights

New Top Tab Functions: 14+ Databases and SNMP

We've added interactive query analysis functions to database collectors, letting you identify slow queries, long-running operations, and performance bottlenecks directly from the Netdata dashboard—no need to connect to the database and run diagnostic queries manually.

Database	Functions	Description
ClickHouse	Top Queries	• Aggregated query stats from system.query_log• Execution time, memory usage, rows read/written
CockroachDB	Top Queries	• Statement statistics from crdb_internal• Execution counts, latency, rows processed
	Running Queries	• Currently executing statements via SHOW CLUSTER STATEMENTS• Client info, duration, distributed execution status
Couchbase	Top Queries	• Completed N1QL requests from system:completed_requests• Service time, result count, error tracking
Elasticsearch/OpenSearch	Top Queries	• Active search tasks from Tasks API• Running time, search type, node distribution
MongoDB	Top Queries	• Slow operations from system.profile• Execution time, docs examined, keys examined, plan summary
MS SQL Server	Top Queries	• Query Store statistics• CPU time, logical reads/writes, memory grants, parallelism
	Deadlock Info	• Latest deadlock from system_health Extended Events• Victim process, lock mode, wait resource
	Error Info	• Recent SQL errors from Extended Events session• Error number, message, query text
MySQL/MariaDB	Top Queries	• Digest statistics from performance_schema• Execution time, lock time, rows examined/sent
	Deadlock Info	• Latest InnoDB deadlock from SHOW ENGINE INNODB STATUS• Victim transaction, lock mode, wait resource
	Error Info	• Recent SQL errors from Performance Schema history• Error number, SQLSTATE, message per query digest
Oracle DB	Top Queries	• SQL statistics from V$SQLSTATS• CPU time, elapsed time, buffer gets, disk reads
	Running Queries	• Active sessions from V$SESSION• Wait events, blocking sessions, SQL text
PostgreSQL	Top Queries	• Statement statistics from pg_stat_statements• Total/mean time, shared blocks hit/read, temp blocks
	Running Queries	• Active queries from pg_stat_activity• Duration, wait events, client info, backend state
ProxySQL	Top Queries	• Query digest from stats_mysql_query_digest• Execution time, rows affected/sent, errors
Redis	Top Queries	• Slow commands from SLOWLOG• Command name, duration, client info
RethinkDB	Running Queries	• Active jobs from rethinkdb.jobs• Query text, duration, involved servers
YugabyteDB	Top Queries	• YSQL statistics from pg_stat_statements• Execution time, calls, rows processed
	Running Queries	• Active backends from pg_stat_activity• Query state, wait events, elapsed time
Generic SQL	User-Defined	• Custom SQL functions defined in job configuration• Interactive table views in Top tab for any SQL database

What You Can Do:

• Identify resource hogs: Sort by total execution time, CPU time, or I/O
• Spot frequent queries: Find high-call-count queries that may benefit from caching
• Debug in real-time: See currently running queries and which client initiated them
• No extra tools needed: All analysis happens in the Netdata dashboard's Top tab

[!IMPORTANT] Some functions require database-specific configuration (e.g., enabling Query Store, pg_stat_statements, or profiling).

See each collector's documentation for prerequisites.

Also Added: SNMP Network Interfaces Function

Collector	Function	Description
SNMP	Network Interfaces	• Real-time interface status and traffic• Packets in/out, errors, discards• Filterable by type (Ethernet, Aggregation, Virtual)

Microsoft SQL Server Collector (Go)

We've rebuilt the Microsoft SQL Server collector from the ground up in Go, replacing the legacy C implementation. This brings comprehensive SQL Server monitoring with real-time query analysis capabilities.

What You Get:

Feature	Details
Full Metric Coverage	Connections, batch requests, buffer manager, memory, wait statistics, locks, per-database transactions, I/O latency, file sizes, and SQL Agent job status
Top Queries Function	Real-time view of your most resource-intensive queries directly in the dashboard—see execution time, CPU, I/O, memory grants, and parallelism metrics
UI Configuration	Create and manage monitoring jobs through the Netdata dashboard—no manual config file editing required
Replication Monitoring	Track replication status, latency, and warnings for transactional/merge publications
Flexible Authentication	SQL authentication, Windows integrated auth, named instances, and remote servers

Top Queries: Identify Performance Bottlenecks

The new Top Queries function pulls data from SQL Server's Query Store to show:

• Queries consuming the most total execution time or CPU
• I/O patterns (logical reads, physical reads, writes)
• Memory grants and tempdb usage
• Parallelism (DOP) across executions

[!IMPORTANT] Requires Query Store enabled on monitored databases (SQL Server 2016+).

Query text may contain sensitive data—ensure proper dashboard access controls.

OpenTelemetry Logs Ingestion

We're expanding our OpenTelemetry support! Following the metrics support added in v2.7.0, the otel plugin now ingests OpenTelemetry logs through the same OTLP gRPC endpoint.

What's New

• Log Ingestion: Send logs from your OpenTelemetry Collector or instrumented applications directly to Netdata.
• Custom Storage: Logs are stored in systemd-compatible journal files using a Rust implementation (no libsystemd dependency required).
• Unified Interface: View OpenTelemetry logs alongside system logs in the Logs tab with full search and filtering capabilities.

Configuration & Retention

Control your log storage with configurable retention policies:

• Set limits by disk usage, file count, or age
• Define custom storage locations
• Tune file size and time span settings

Availability

Available on Linux agents (beta), with support for additional platforms coming soon.

---

[!NOTE]
This feature is in beta. We welcome your feedback!

Contributions

Collectors

• Added dynamic configuration support to go.d Service Discovery with UI-driven templates (#21680, #21718, #21730, @ilyam8)
• Added SNMP profile metadata mapping for “Westermo Teleindustri AB” to “Westermo” (#21727, @ilyam8)
• Added a dcgm collector for NVIDIA dcgm-exporter with full field classification (#21721, @ktsaou)
• Added jitter and variance metrics to the go.d ping collector (#21683, @ktsaou)
• Added a Kubernetes API Server collector (#21682, @ktsaou)
• Added function support to the go.d SQL collector to enable interactive, filterable table views in the Top tab (#21666, #21671, #21678, @ilyam8)
• Added PostgreSQL running-queries and enhanced top-queries support with pg_stat_monitor detection (#21656, @ktsaou)
• Added Windows support to hardware monitoring collectors with direct vendor CLI execution and improved reliability (#21635, @ktsaou)
• Added new cloud functions for MySQL and MSSQL to provide error and deadlock information (#21645, #21650, #21652, @ktsaou, @ilyam8)
• Added smart ZFS dataset deduplication to the diskspace plugin to reduce duplicate reporting (#21643, @ilyam8)
• Added top-queries and running-queries functions for nine additional database collectors: Redis, ClickHouse, Elasticsearch, Couchbase, ProxySQL, OracleDB, CockroachDB, YugabyteDB, and RethinkDB (go.d). (#21607, @ktsaou)
• Added interface type categorization to SNMP collector for easier filtering (go.d/snmp). (#21605, @ilyam8)
• Added interfaces function for SNMP collector with per-interface traffic, packet, error/discard, and status metrics with computed rates (go.d/snmp). (#21604, @ilyam8)
• Implemented Top Queries Functions framework for PostgreSQL, MySQL, MSSQL, and MongoDB with a reusable Functions API for building Netdata Functions in Go (go.d). (#21595, @ktsaou)
• Added a new Microsoft SQL Server (MSSQL) collector to go.d.plugin with comprehensive monitoring of connections, transactions, replication, and more (go.d/mssql). (#21583, @ktsaou)
• Added hostgroup-level summary metrics and hostgroup label to all ProxySQL backend charts (go.d/proxysql). (#21549, @ilyam8)
• Added ProxySQL health alerts for SHUNNED/OFFLINE_HARD backends and hostgroups with no ONLINE backends (go.d/proxysql). (#21548, @ilyam8)
• Added IPC mutexes chart for Windows systems (windows.plugin). (#21474, @thiagoftsm)
• Added journal-viewer plugin replacing the existing systemd-journal plugin with improved OpenTelemetry logs ingestion in a systemd-compatible way (journal-viewer-plugin). (#21356, #21561, #21568, #21578, #21581, #21625, @vkalintiris)
• Added EATON UPS SNMP profile for automatic discovery and collection of battery, power, and status metrics (go.d/snmp). (#21355, @ilyam8)
• Added four new Windows sensors: human presence, human proximity, electrical capacitance, and inductance (windows.plugin). (#21319, @thiagoftsm)
• Added a scripts.d plugin to run unmodified Nagios plugins inside Netdata (#21294, #21759, #21774, @ktsaou)

• Fixed systemd-units.plugin response content type to application/json (systemd-units.plugin). (#21621, @ilyam8)
• Fixed Windows plugin metric names, error logging, and cleanup hooks (windows.plugin). (#21597, @thiagoftsm)
• Fixed ProxySQL collector to use timeout when pinging instances (go.d/proxysql). (#21573, @ilyam8)
• Fixed go.d logger to disable terminal color on Windows (go.d). (#21562, @ilyam8)
• Fixed cgroup duplicate detection to only treat a match as a duplicate when the existing cgroup is enabled and available (cgroups.plugin). (#21559, @ilyam8)
• Fixed SNMP collector nil dereference when no profiles are found (go.d/snmp). (#21557, @ilyam8)
• Fixed Windows plugin driver lifecycle and CPU temperature collection thread safety (windows.plugin). (#21523, @thiagoftsm)
• Fixed eBPF cleanup by properly cleaning up shared memory and semaphores (ebpf.plugin). (#21501, @stelfrag)
• Fixed Windows storage collector memory leaks with dictionary delete callbacks (windows.plugin). (#21500, @stelfrag)
• Fixed MSSQL ODBC bindings and cursor handling for improved collection stability (windows.plugin). (#21478, @thiagoftsm)
• Fixed Windows plugin metrics, error logging, and cleanup across multiple collectors (windows.plugin). (#21474, @thiagoftsm)
• Fixed Windows plugin service enumeration and chart correctness (windows.plugin). (#21466, @thiagoftsm)
• Fixed incorrect perflib mappings, labels, and priorities in Windows plugin collectors (windows.plugin). (#21458, @thiagoftsm)
• Fixed MSSQL NULL result handling to prevent bad metrics in replication scenarios (windows.plugin). (#21417, @thiagoftsm)

• Improved panic handling by always capturing and logging backtraces in agent logs (journal-viewer) (#21681, @vkalintiris)
• Added function-only mode to the go.d plugin, allowing modules and jobs to expose functions without collecting metrics (#21646, @ilyam8)
• Refactored the go.d function API for cleaner, more consistent, and more maintainable collector code (#21633, #21655, @ilyam8)
• Added RequireCloud flag for database functions to restrict sensitive query data to cloud-only access (go.d). (#21629, @ilyam8)
• Set default update interval to 10 seconds for all go.d collector functions (go.d). (#21616, @ilyam8)
• Skipped internal SQL Server databases (model_*) in MSSQL go.d collector (go.d/mssql). (#21588, @ilyam8)
• Removed per-metric collect_* options from MSSQL go.d collector - all metrics now always collected (go.d/mssql). (#21586, @ilyam8)
• Added option to trust MSSQL server certificates in Windows plugin for easier TLS setup (windows.plugin). (#21582, @thiagoftsm)
• Migrated pgbouncer collector to jackc/pgx/v5 (go.d/pgbouncer). (#21510, @ilyam8)
• Merged MSSQL queries to reduce runtime needed for metrics collection (windows.plugin). (#21491, @thiagoftsm)
• Added MSSQL blocked processes collection using SQL query with configurable option (windows.plugin). (#21426, @thiagoftsm)
• Formatted MSSQL Windows collector code to match project style (windows.plugin). (#21383, @thiagoftsm)
• Added support for float-type dimensions in go.d collectors (go.d). (#21362, #21363, @ilyam8)
• Added MSSQL session connections chart alongside user connections (windows.plugin). (#21350, @thiagoftsm)
• Added MSSQL user connections collection via SQL query for improved reliability over Perflib (windows.plugin). (#21348, @thiagoftsm)
• Reorganized MSSQL perflib code for improved readability (windows.plugin). (#21334, @thiagoftsm)

Packaging/Installation

• Restored the C-based systemd-journal.plugin (#21729, @vkalintiris)
• Updated bundled libbpf to v1.6.3.1p_netdata (#21717, @thiagoftsm)
• Simplified LZ4 detection by requiring liblz4 ≥ 1.7.1 and enabling LZ4 builds consistently (#21669, @Ferroin)
• Fixed $releasever_major usage to only apply on RHEL 9+, keeping RHEL 8 on $releasever. (#21610, #21649, @Ferroin)
• Fixed repo URL to use only major version for RHEL 9/10. (#21596, @Ferroin)
• Updated Azure signing action after rebranding for Windows builds. (#21591, @ilyam8)
• Fixed 32-bit builds of the journal-viewer plugin. (#21580, @vkalintiris)
• Enabled ARM package builds in PR CI runs for better testing coverage. (#21570, @Ferroin)
• Restored correct permissions and capabilities for systemd-journal.plugin during source installs. (#21569, @ilyam8)
• Removed Ubuntu 25.04 from CI and package builds after upstream EOL. (#21519, @Ferroin)
• Fixed major version detection for native packages in updater. (#21485, @ilyam8)
• Removed Fedora 41 from CI and package builds after upstream EOL. (#21475, @Ferroin)
• Added automatic repair for systems affected by the /dev/null deletion bug in the updater. (#21432, @Ferroin)
• Fixed RPM updater check URL generation for RHEL-family distros. (#21431, @Ferroin)
• Fixed updater bug that could delete /dev/null when URL check fails. (#21428, @ilyam8)
• Fixed EPEL installation order to prevent failed installs on EL systems. (#21410, @ilyam8)
• Fixed incorrect find command patterns in kickstart, causing installation failures. (#21399, @ilyam8)
• Added concurrency to the check-markdown workflow to prevent duplicate runs. (#21393, @ilyam8)
• Added a check to report when updates fail due to native packages no longer being published. (#21288, @Ferroin)
• Added intelligent fallback behavior in the kickstart script based on native package availability detection. (#21262, @Ferroin)
• Implemented systemd-tmpfiles for handling required directories at runtime. (#21243, @Ferroin)
• Minor improvements to CMake code following best practices. (#21146, @Ferroin)
• Suppressed known build warnings for cleaner build output. (#20102, @Ferroin)

Documentation

• Fixed incorrect icon filenames in integration metadata so icons render correctly (#21767, @Ancairon)
• Fixed metadata validation for ibm.d, network-viewer, and netdata-otel (#21756, @ktsaou)
• Updated documentation to reference Cloud MCP alongside Agent and Parent MCP (#21746, @ktsaou)
• Restructured integration categories, fixed related_resources, added guides (#21742, #21749, #21751, #21753, @ktsaou)
• Improved cgroups.plugin discoverability by organizing it into per-technology modules (#21737, #21740, @ktsaou)
• Added the missing type = api to streaming parent configuration examples (#21739, @ktsaou)
• Added documentation for the Netdata Cloud MCP server (#21736, @juacker)
• Added clarification to Plan & Billing docs about when cancellations take effect (#21728, @kanelatechnical)
• Added documentation recommending bearer token protection as the preferred security method (#21712, @ktsaou)
• Added documentation for opting out of Windows telemetry (#21710, @kanelatechnical)
• Added documentation for logs support in the OpenTelemetry plugin (#21705, @vkalintiris)
• Added documentation for access control and feature availability (#21703, @ktsaou)
• Fixed relative Markdown links to ensure proper ingestion by the Learn site (#21688, #21689, #21690, @ktsaou)
• Improved Netdata architecture description (#21642, @ktsaou)
• Added node-states-and-transitions to learn map (#21631, @ktsaou)
• Added reporting documentation page (#21520, @ktsaou)
• Added comprehensive node states and transitions documentation covering Live, Stale, Offline, and Unseen states with timing details. (#21630, @ktsaou)
• Improved go.d collector functions metadata with version-aware descriptions. (#21626, @ilyam8)
• Updated collector function docs, schema, and templates for improved clarity. (#21622, @ilyam8)
• Added collector function documentation with metadata for Top Queries/Running Queries across 14 collectors. (#21618, @ilyam8)
• Clarified funcapi types and enums with explicit documentation comments. (#21617, @ilyam8)
• Removed obsolete TODO documentation files. (#21614, @ilyam8)
• Added alert configuration ordering and override documentation with the conceptual model. (#21613, @ktsaou)
• Updated kickstart.md note formatting. (#21600, @kanelatechnical)
• Added documentation for Real-Time Conversations with live exhibits and updated AI docs structure. (#21594, @shyamvalsan)
• Added On-Prem Release Notes to documentation map. (#21571, @Ancairon)
• Added ProxySQL alerts to metadata.yaml for documentation. (#21565, @ilyam8)
• Added SOC 2 Type 2 compliance badge to Security and Privacy documentation. (#21554, @Ancairon)
• Added reciprocal links between ephemerality, identity, and template documentation. (#21552, @ktsaou)
• Fixed broken links in vm-templates.md and various documentation files. (#21546, @ilyam8)
• Added unclaim/reclaim node guide for moving agents between Spaces. (#21539, @kanelatechnical)
• Added VM templates and cloning guide for preparing Netdata installations for cloning. (#21527, @ktsaou)
• Cleaned up go.d/prometheus metadata by removing archived and unmaintained repositories. (#21498, @ilyam8)
• Added AI-Powered Alert Configuration documentation for creating alerts in plain English. (#21482, @kanelatechnical)
• Added automatic node instance cleanup documentation to nodes-ephemerality page. (#21471, @kanelatechnical)
• Added "docker" keyword to the cgroups plugin for search discoverability. (#21469, @Ancairon)
• Clarified Docker image tags: netdata/netdata defaults to latest, use :stable for stable releases. (#21467, @ilyam8)
• Added troubleshooting guide for SNMP chart gaps with timing diagnostics. (#21452, @ilyam8)
• Fixed admonition syntax in SNMP documentation. (#21445, @ilyam8)
• Removed third-party network device Prometheus exporters from documentation. (#21443, @ilyam8)
• Added keywords and improved the overview for SNMP documentation. (#21442, @ilyam8)
• Updated PLATFORM_SUPPORT.md with current distro versions and static build details. (#21422, @kanelatechnical)
• Removed Docker prerequisite to install smartmontools as it's now included in helper images. (#21413, @ilyam8)
• Added keyword support to the documentation map for improved search relevance. (#21402, @Ancairon)
• Updated Home tab documentation to reflect the current Netdata Cloud UI. (#21386, @kanelatechnical)
• Removed snippet with broken link from go_expvar metadata. (#21382, @Ancairon)
• Fixed dangling link to sizing documentation. (#21372, @ralphm)
• Updated security and privacy design links in map.csv. (#21369, @ilyam8)

Other Notable Changes

• Ensured dyncfg-generated alerts fully sync to Cloud on creation and prevented duplicate configuration sends (#21706, @stelfrag)
• Improved agent event posting stability by preventing unintended stdout writes and disabling signal-based timeouts (#21696, @stelfrag)
• Improved statement preparation with spinlock protection to avoid shutdown crashes. (#21608, @stelfrag)
• Used preallocated static buffer when writing daemon status during shutdown. (#21593, @stelfrag)
• Added health database check before processing alerts and saving logs. (#21592, @stelfrag)
• Fixed MCP test client prompts/resources support and schema validation. (#21521, @ktsaou)
• Improved SSL certificate verification error logging with better memory handling. (#21513, @stelfrag)
• Improved ACLK query execution with better error handling and resource cleanup. (#21479, @stelfrag)
• Improved metadata storage stability with better JudyL error handling and vacuum throttling. (#21468, @stelfrag)
• Sent complete node info when node is marked ephemeral for accurate Cloud data. (#21456, @stelfrag)
• Improved streaming connection loss detection with TCP keepalive and dead connection detection. (#21430, @stelfrag)
• Improved thread join handling to prevent race conditions during shutdown. (#21421, @stelfrag)
• Skipped retention check during datafile initialization to speed up startup. (#21387, @stelfrag)
• Added option to configure custom API URL for Telegram notifications. (#21376, @ilyam8)

• Fixed a timeout race that could prematurely close long-running initial web requests (#21722, @ktsaou)
• Fixed shutdown crash by safely handling destroyed dictionaries during function execution (#21713, @stelfrag)
• Fixed ML model loading to correctly map columns and prevent incorrect k-means contexts (#21700, @stelfrag)
• Fixed buffer overflow in the UTF-8 text sanitizer to prevent crashes on invalid or partial input (#21698, @stelfrag)
• Fixed a segmentation fault during datafile extent flush by correcting UUID reference handling (#21667, @stelfrag)
• Fixed claim script newline handling to ensure valid configuration files across different shells (#21665, @ktsaou)
• Fixed Matrix notifications to comply with the Matrix API and prevent request errors (#21659, @ktsaou)
• Fixed CPU frequency detection on FreeBSD to prevent NaN values in system info (#21658, @ktsaou)
• Fixed race condition during thread exit cleanup. (#21603, @stelfrag)
• Fixed function error message key to use camelCase for UI compatibility. (#21602, @ilyam8)
• Fixed potential NULL dereference when releasing metric buffers. (#21615, @stelfrag)
• Fixed race condition on shutdown by properly initializing discovery_thread. (#21563, @stelfrag)
• Fixed crash during ML predictions by acquiring lock before accessing dimension data. (#21555, @stelfrag)
• Fixed incorrect column index for old_value in SQLite health query. (#21533, @stelfrag)
• Fixed health alert db lookup parser with correct operator mappings (< to LESS, > to GREATER) and added == operator support. (#21529, @ktsaou)
• Fixed metric page list retrieval with UUID null check and bounds validation. (#21575, @stelfrag)
• Fixed netdata_mutex_destroy macro to call correct debug function. (#21579, @stelfrag)
• Fixed memory leaks in ACLK error paths for batch processing and alert configuration. (#21541, @stelfrag)
• Fixed journal v2 migration to safely handle deleted metrics and null handles. (#21514, @stelfrag)
• Fixed race condition during journal file deletion by unmapping before deleting. (#21512, @stelfrag)
• Fixed ACLK MQTT send path to skip garbage-collected fragments preventing crashes. (#21483, @stelfrag)
• Fixed functions event loop to propagate exit code instead of calling exit() directly. (#21455, @stelfrag)
• Fixed MQTT packet ACK handling to prevent use-after-free. (#21416, @stelfrag)
• Fixed service shutdown detection during health initialization for faster shutdown. (#21329, @stelfrag)

• Fixed log queries to skip internal field-remapping entries, preventing incorrect field names from appearing in results (#21755, @vkalintiris)
• Improved function execution by centralizing timeout and progress handling in the agent (#21723, @vkalintiris)
• Improved the “deferred response too big” log with detailed context including size, limit, plugin, and transaction ID (#21720, @vkalintiris)
• Downgraded inotify name-modify logs to info to reduce false error noise during systemd file rotation (#21719, @vkalintiris)
• Added configurable indexing limits to prevent memory spikes from large journals and reduced log noise (#21716, @vkalintiris)
• Ensured journal files are marked as archived on shutdown and added graceful signal handling to prevent unnecessary re-indexing (#21707, @vkalintiris)
• Added strict validation for journal object reads to prevent out-of-bounds access and improved writer updates for consistency (#21697, @vkalintiris)
• Fixed journal handling to avoid misclassifying active systemd files as archived during transient state changes (#21695, @vkalintiris)
• Improved journal window management by handling mmap failures safely and tightening error handling (#21693, @vkalintiris)
• Improved panic handling by always capturing and logging backtraces in agent logs (#21681, @vkalintiris)
• Removed redundant libuv loop calls to improve event loop stability and reduce overhead (#21657, @stelfrag)
• Removed Sentry error reporting from journal-viewer plugin (#21612, @vkalintiris)

Deprecation notice

Changed in this release

Important Changes in Next Minor Release

MS SQL Server Collector: C Version Removal

The legacy C-based MS SQL Server collector (part of windows.plugin) will be removed in the next minor release. The Go-based collector (go.d/mssql) is now the default and recommended option.

The Go version offers significant advantages:

• More features — includes the new Top Queries function for real-time query analysis.
• Multi-job support — monitor multiple SQL Server instances from a single agent.
• Remote connections — connect to SQL Server instances over the network, not just localhost.
• UI configuration — create and manage monitoring jobs directly from the Netdata dashboard.
• Easier maintenance — benefits from the go.d framework's shared infrastructure and faster development cycles.

If you're currently using the C collector, see the Go collector setup documentation for configuration instructions.

Important Changes in Next Major Release

Deprecated Components

Component Type	Versions Being Deprecated
APIs	v1, v2

What This Means

Only the v3 API and v3 Dashboard will be supported starting with the next major release. These newer versions offer improved performance, enhanced features, and better security.

Support options

As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:

• Premium Support: Customers who wish to have a direct channel with Netdata and prioritized support with defined SLAs can contact us.
• Netdata Learn: Find documentation, guides, and reference material for monitoring and troubleshooting your systems with Netdata.
• GitHub Issues: Use the Netdata repository to report bugs or open a new feature request.
• GitHub Discussions: Join the conversation around the Netdata development process and be a part of it.
• Community Forums: Visit the Community Forums and contribute to the collaborative knowledge base.
• Discord Server: Jump into the Netdata Discord and hang out with like-minded sysadmins, DevOps, SREs, and other troubleshooters. More than 2000 engineers are already using it!

Netdata v2.8.5 is a patch release to address issues discovered since v2.8.4.

This patch release provides the following bug fixes and updates:

• Updated the Netdata Docker image to Debian 13.3 with the latest upstream updates and security fixes
• Standardized alert configuration to use a consistent type for time group values (#21528, @ilyam8)
• Fixed CSV parser configuration serialization so empty settings are correctly omitted from logs configuration output (#21526, @ilyam8)
• Fixed ProxySQL backend status metrics in the go.d collector to correctly reflect reported backend states (#21524, @ilyam8)
• Fixed proc file parsing to support non-seekable files, restoring network stats collection on affected kernels (#21507, @ilyam8)
• Fixed edit-config to ignore inherited container environment values, preventing incorrect command handling (#21505, @ilyam8)
• Fixed vnode label configuration by allowing arbitrary label keys in the go.d schema, enabling labels to be set via the UI (#21503, @ilyam8)

Support options

As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:

• Netdata Learn: Find documentation, guides, and reference material for monitoring and troubleshooting your systems with Netdata.
• GitHub Issues: Make use of the Netdata repository to report bugs or open a new feature request.
• GitHub Discussions: Join the conversation around the Netdata development process and be a part of it.
• Community Forums: Visit the Community Forums and contribute to the collaborative knowledge base.
• Discord Server: Jump into the Netdata Discord and hang out with like-minded sysadmins, DevOps, SREs, and other troubleshooters. More than 2000 engineers are already using it!

Netdata v2.8.4 is a patch release to address issues discovered since v2.8.3.

This patch release provides the following bug fixes and updates:

• Include Go version in build info (#21473, @ilyam8)
• Update go toolchain to v1.25.5 (#21459, @ilyam8)

Support options

As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:

• Netdata Learn: Find documentation, guides, and reference material for monitoring and troubleshooting your systems with Netdata.
• GitHub Issues: Make use of the Netdata repository to report bugs or open a new feature request.
• GitHub Discussions: Join the conversation around the Netdata development process and be a part of it.
• Community Forums: Visit the Community Forums and contribute to the collaborative knowledge base.
• Discord Server: Jump into the Netdata Discord and hang out with like-minded sysadmins, DevOps, SREs, and other troubleshooters. More than 2000 engineers are already using it!

Netdata v2.8.3 is a patch release to address issues discovered since v2.8.2.

This patch release provides the following bug fixes and updates:

• Fixed the go.d AP collector to handle unknown station statistics gracefully (#21461, @ilyam8)
• Fixed incorrect plugin and binary paths on Windows for the go.d plugin by correctly detecting the installation prefix at startup (#21451, @ilyam8)
• Improved go.d job management by avoiding blocking during job shutdown, keeping other jobs responsive (#21448, @ilyam8)
• Fixed and standardized Windows AD, ADCS, and ADFS charts, and improved stability and accuracy of Windows hardware metrics collection (#21433, #21454, @thiagoftsm)
• Fixed SNMP service discovery by accepting version “2” and using the default polling interval instead of overriding it (#21424, @ilyam8)
• Logged data collection duration when go.d collections are skipped due to a previous run still in progress (#21423, #21425, @ilyam8)
• Fixed the MSSQL errors chart in the Windows plugin by using the correct metric (#21412, @thiagoftsm)
• Removed the strict rabbitmq_version check to improve RabbitMQ collector compatibility with older brokers (#21411, @ilyam8)
• Added collection statistics to the SNMP go.d collector and exposed them as per-profile charts (#21409, @ilyam8)
• Updated ndexec runner to return captured stdout on command failures (#21405, @ilyam8)
• Disabled the MongoDB exporter on affected Ubuntu versions due to insecure libbson dependencies (#21403, @Ferroin)
• Updated bundled components used in static builds (#21401, @Ferroin)
• Fixed off-by-one validation in Journal v2 by treating extent_index == extent_entries as invalid, preventing out-of-bounds access (#21400, @stelfrag)
• Fixed timed waits in completion to handle spurious wakeups and honor shutdown timeouts, preventing premature timeouts and hangs (#21395, @stelfrag)
• Prevented datafiles from exceeding their max size by making size checks atomic and accounting for incoming writes (#21390, @stelfrag)
• Updated DEB package builds to use XZ or zstd compression instead of gzip (#21310, @Ferroin)

Support options

As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:

• Netdata Learn: Find documentation, guides, and reference material for monitoring and troubleshooting your systems with Netdata.
• GitHub Issues: Make use of the Netdata repository to report bugs or open a new feature request.
• GitHub Discussions: Join the conversation around the Netdata development process and be a part of it.
• Community Forums: Visit the Community Forums and contribute to the collaborative knowledge base.
• Discord Server: Jump into the Netdata Discord and hang out with like-minded sysadmins, DevOps, SREs, and other troubleshooters. More than 2000 engineers are already using it!

Netdata v2.8.2 is a patch release to address issues discovered since v2.8.1.

This patch release provides the following bug fixes and updates:

• Adjusted Windows sensors initialization by moving COM and Sensor API setup into the sensors thread for better stability (#21374, @stelfrag)
• Replaced dots with slashes in OTEL metric families to enable hierarchical grouping in dashboards (#21371, @ilyam8)
• Fixed user group updates to ensure proper Proxmox group assignment in Docker entrypoint (#21364, @ilyam8)
• Ensured Netdata has access to NVIDIA device files by adding the netdata user to the appropriate group, fixing GPU monitoring on non-Debian systems (#21358, #21359, @ilyam8)
• Prevented replication from stalling by detecting empty-response loops and safely completing the process (#21357, @stelfrag)
• Stopped replication when the parent is already caught up, preventing stalls and unnecessary gap-filling (#21352, @stelfrag)
• Prioritized environment-provided config directories in go.d for runtime overrides (#21345, @ilyam8)
• Silenced Redis client library logs in the go.d Redis collector to reduce noise (#21344, @ilyam8)

Support options

As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:

• Netdata Learn: Find documentation, guides, and reference material for monitoring and troubleshooting your systems with Netdata.
• GitHub Issues: Make use of the Netdata repository to report bugs or open a new feature request.
• GitHub Discussions: Join the conversation around the Netdata development process and be a part of it.
• Community Forums: Visit the Community Forums and contribute to the collaborative knowledge base.
• Discord Server: Jump into the Netdata Discord and hang out with like-minded sysadmins, DevOps, SREs, and other troubleshooters. More than 2000 engineers are already using it!