exo

EXO v1.0.71 Release Notes

This is a small patch release, with better defaults for sampling and bug fixes for M5 series Macs and RDMA, along with support for the new Kimi K2.6.

Models

• Add Kimi K2.6, including support for multimodality (#1952)
• Add model cards for Qwen3.6 27B, GLM 5.1 and Kimi K2.6 quantisations (#1970)

Performance

• Add recommended sampling defaults (from model providers) to improve model output quality, including min_p and top_k (#1947)
• Prevent JACCL all_sum from corrupting output (#1952)

Quality of Life

• Add a Pi integration tab! (#1925)
• Handle multiple tool calls correctly in the Claude and Responses APIs (#1952)
• Add a friendly context message for bug reporting (#1959)

Bug Fixes

• Fix vision models on M5 series Macs by fixing addmm on NAX (#1927)
• Fix prefix cache for Qwen3.6 (#1952)
• Prevent complete downloads from losing their Completed status (#1956, #1958)
• Populate the image cache from events to prevent crashes (#1961)
• Fix DeepSeekV32 parsing and tensor parallelism for Gemma 4 models with effective parameters (#1967)

New Contributors

And a thank you to @nhwaani who made their first contribution here!

Full Changelog: https://github.com/exo-explore/exo/compare/v1.0.70...v1.0.71

EXO v1.0.70 Release Notes

This release ships with multimodality and major enhancements for memory usage in long context use cases (e.g. OpenClaw and OpenCode), as well as updated model support and QOL features.

Models

• Add support for Gemma 4 (#1851, #1891)
• Add support for Minimax M2.7 (#1884)
• Add support for Qwen3.6 (#1907, #1917)

Performance

• Implement Flash Attention for Qwen3.5 and Gemma 4, reducing peak memory consumption by 3-6x (#1886)
• Fix memory leaks in Rotating and Arrays cache (#1819, #1835)
• Improve KV prefix cache hit rates (#1817)
• Garbage collect KV cache on eviction (#1832)

Multimodality

• Add support for vision models, making use of vision processors in Qwen3.5, Kimi K2.5 and Gemma 4 (#1802, #1893)
• Handle PDFs by sending both text and image (#1822, #1865)

API

• Add integration helpers to display one-click setups for connecting to tools such as OpenCode, n8n and OpenClaw (#1810)
• Add Firefox AI sidebar (?q=) support to dashboard (#1814)
• Expose /state/paths for easier lookup of pieces of state (#1796)
• Improve stats and usage reporting (#1836, #1848, #1885, #1899)
• Prevent timing out on clients during long prefills (#1803, #1828)
• Allow copying even on http, as a QOL for Tailscale users and other non localhost serving (#1902)

Quality of Life

• Allow pausing and deleting active downloads (#1829)
• Add the ability to edit environment variables in MacOS app (#1852, #1869, #1901)
• Prevent failed instance retry loops (#1763)
• Truncate overly verbose bits of text in logs (#1854)

Bug Fixes

• Clean up processes nicely to prevent zombie processes and holding RDMA resources (#1889)
• Fix out of order event crashes on startup (#1894)
• Drain tokens silently skipped in thinking parsing, prevent malformed tool calls in certain models (#1898)
• Gracefully catch errors in downloads (#1846)
• Prevent several fatal crashes (#1856, #1827)
• Use configured api_port for IP connectivity probes (#1877)

New Contributors

We had some more new faces in this release. Thank you for your contributions!

• @kaiisfree
• @MikkoParkkola
• @lawrence3699
• @adurham

Full Changelog: https://github.com/exo-explore/exo/compare/v1.0.69...v1.0.70

EXO v1.0.69 Release Notes

This release ships with continuous batching, Qwen3.5 support and support for M5 Pro/Max chips, as well as a host of quality of life improvements and bug fixes.

Continuous batching is on by default, enabling you to run multiple requests in parallel for significantly higher throughput. EXO will automatically batch together inference requests, on single node and multi-node instances including RDMA instances. This is particularly useful for agentic workflows where multiple agents can run in parallel.

Models

• Add support for Qwen3.5 (#1644)
• Add support for Nemotron sharding (#1693)
• Add default model cards for DeepSeek v3.2 (#1769)

API

• Add POST /v1/cancel/{command_id} endpoint for cancelling ongoing text generations (#1579)
• Add reasoning params to chat completions and responses APIs (#1654)
• Add repetition_penalty and repetition_context_size to chat completions (#1665)

Performance

• Continuous batching (#1642, #1632, #1777)
• Better pipeline parallel prefill that splits the prompt into chunks and overlaps computation and communication. This makes pipeline parallel prefill up to 1.98x faster on 2 nodes (#1587, #1629)

Quality of Life

• Support trace deletion in dashboard (#1628)
• Make mini topology sidebar navigate to home on click (#1616)
• Show feedback that the model was successfully added when adding custom models from Huggingface (#1661)
• Enable global model search from huggingface, not just mlx-community (#1661)
• Mobile-friendly UI (#1677)
• Include power usage in exo-bench responses, enabling benchmarks to capture energy usage (#1693)
• Prefer nodes with more of the model downloaded when placing an instance (#1767, #1795)
• Sync custom model cards across nodes (#1768)
• Add --bootstrap-peers and --libp2p-port for static peer discovery, bypassing mDNS which Is useful in environments where mDNS is unavailable (#1690)

Image Generation (Experimental)

• Update mflux to 0.16.9 (#1751)
• Support image generation cancellation (#1774)

Bug Fixes

• Upgrade macmon to fix macmon errors on M5 Pro/Max. This fixes an issue where M5 Pro/Max would not report memory or GPU usage stats and therefore could not participate in clusters (#1747, #1797)
• Use tmpdir for MLX distributed coordination file, preventing local network access permission issues (#1624)
• Fix BrokenResourceError crash when immediately loading a model on start (#1637)
• Emit error chunks when a runner crashes in the middle of a request, preventing streams hanging forever when runners crash (#1645)
• Fix copy code button not working in dashboard (#1659)
• Fix re-downloads so that models can be downloaded again after being deleted via the dashboard (#1658)
• Reset download status to DownloadPending when a download is cancelled so that the API and dashboard reflect the correct model download status (#1674)
• Increase gossipsub message limit to 8MB, fixing requests with very large prompts (#1671)
• Clean up stale state.runners state when runners shut down (#1684)
• Fix emoji rendering in chat responses (#1691)
• Fix placement validation for tensor sharding and show an error message with the constraints when no valid placement is found (#1669)
• Normalise responses API tool call format. This fixes tool calling with n8n (#1704)
• Show partial download progress on initial dashboard load (#1706)
• Fix a runner crash where very fast requests would cause a ZeroDivisionError (#1707)
• Make prefill more consistent on slow machines (#1748)
• Fix a race condition during master reelection that caused a full node hang with IOConnectUnmapMemory failed: kr=0xe00002bc (#1801)
• Use mlx_generate for warmup, preventing occasional issues with warmup when using stream_generate (#1794)
• Fix DeepSeek v3.2 warmup crash and tool calling (#1769)

New Contributors

Thank you to everyone who made contributions to exo for the first time:

• @onel
• @nakheel77
• @asprooo
• @zeus247
• @sigkill
• @EthyMoney
• @Luckystrike561
• @saidulahmed2266-cloud
• @tiawl

Full Changelog: https://github.com/exo-explore/exo/compare/v1.0.68...v1.0.69

EXO v1.0.68 Release Notes

This is the biggest EXO release to date. We wanted to make sure we address the stability issues users were running into on previous versions and we think we've achieved that with this release. This release also comes with a whole load of new features and UX improvements, full list below. Thank you to everyone who submitted bug reports over the past few weeks - it helps us to improve EXO much faster.

Models

• Add support for custom models from Huggingface (#1368)
• Add support for Qwen3-Coder-Next (#1367)
• Add support for Step 3.5 Flash (#1460)
• Add support for GLM 5 (#1526), (#1529)
• Add support for MiniMax M2.5 (#1514)

API

• Add support for Claude Messages API, enabling tools like Claude Code (#1167)
• Add support for OpenAI Responses API (#1167)
• Add usage and generation stats to API, enabling clients like OpenCode to consume stats including prompt tokens, completion tokens and total tokens (#1333), (#1461)
• Cancel text generation when API request is closed (#1276)
• Add support for Ollama API (#1560)

Web Dashboard

• Add redesigned model picker modal (#1369), (#1377), (#1440), (#1470)
• Display alternative tokens / logprobs visualizer in chat responses (#1180)
• Redesign downloads page as model x node table (#1465), (#1589), (#1581)
• Add prefill progress bar for long prompts (#1181), (#1557)
• A new onboarding flow when running EXO for the first time (#1533)
• Automatic model selection / model recommendations in web dashboard (#1590)

Quality of Life

• Show a more informative message in macOS app when installing network location (#1309)
• Migrate model cards to .toml files (#1354)
• Clean up exo gracefully on shutdown, preventing memory not being cleaned up on exit (#1388)
• Make topology updates more responsive by yielding from reachability checks instead of waiting for all checks (#1427)
• Allow typing in chat input while response is generating (#1433)
• Add log rotation, now exo logs get written to ~/.exo/exo_logs (#1438), (#1439), (#1442)
• Distinguish between model fits in available memory and fits in total memory in model picker (#1441 h/t @Hmbown)
• Add enable_thinking toggle for models that support thinking/non-thinking (#1457)
• Show a warning in the web dashboard when macOS versions of nodes in a cluster are incompatible (#1436)
• Show macOS version in debug mode on web dashboard
• Add cancellation button and cancel during prefill (#1540), (#1575)
• Strip Claude headers to improve prefix cache hit rates (#1552)
• Prioritise Thunderbolt for Ring (TCP/IP) instances (#1556)
• Show paused downloads with completion % in web dashboard (#1564)
• Add support for loading models from arbitrary paths with EXO_MODELS_PATH environment variable (#1574)

Image Generation (Experimental)

• Add support for non-streaming image generation (#1328)
• Add support for parallel classifier-free guidance (CFG) for Qwen image models, embarrassingly parallel image generation with improved performance (#1361)
• Add more image dimensions for image generation (1024x1365 and 1365x1024) (#1395)
• Add support for FLUX.1-Kontext-dev, an image editing variant of FLUX.1-dev (#1394)

Bug Fixes

• Fix GPU locks, caused by MLX_METAL_FAST_SYNCH. This would often cause nodes to be stuck at 100% GPU utilization where the only fix is to reboot the machine (#1429), (#1489), (#1515)
• Fix Pipeline instances crashes when MLX_METAL_FAST_SYNCH was enabled (previously only RDMA, now also Ring instances) (#1620), (#1622))
• Fix ConfigData validation for kimi-k2, fixing loading issues (#1314)
• Fix uninstall button not working in some cases (#1306)
• Load pipeline layers sequentially, preventing instances getting stuck in LOADING (#1329)
• Make node ids unique per session, fixing double identities when users copy their entire ~/.exo folder to another node (#1338)
• Skip duplicate tasks in worker, preventing tasks from getting processed twice causing instances to get stuck sometimes (#1342), (#1381)
• Fix out of sync issues with prefix caching on multiple nodes (#1341)
• Remove mDNS discovered peers from appearing in state, fixing issues with broken one-sided connections (#1312)
• Create config home when checking for config file, fixing crashes on Linux systems (#1353)
• Fix tool calling edge case when max_tokens truncates generation mid-tool-call, causing request to hang (#1344)
• Fix Kimi tool calling id, fix GPT-OSS tool calling bug (#1413), (#1487), (#1529)
• Ensure EXO works with no internet (#1363), (#1402)
• Cancel downloads for deleted instances, preventing downloads from continuing silently in the background (#1393)
• Retry transient download errors (#1398)
• Move event log from unbounded in-memory list to disk, fixing a memory leak (#1432)
• Never save to app directory, preventing local network access permission from being silently lost requiring users to manually toggle this in System Settings (#1435)
• Fix tensor parallel sharding for MiniMax, Qwen3Next, Qwen3MoE and GLM4MoE models (#1318), (#1411), (#1595), (#1604)
• Slow down catchup, preventing too large message being sent over Gossipsub for long-running clusters. This speeds up and improves stability when nodes join or re-join long running clusters (#1407)
• Fix setrlimit crash when hard file descriptor limit < 65535 (#1430 h/t @mustafalpyilmaz)
• Fix RDMA debug info in debug mode on web dashboard (#1437)
• Prevent DownloadModel event flood, fixing issues with event logs growing rapidly, causing cluster to get stuck (#1452)
• Don't time out node identifies, preventing nodes getting stuck as a hexagon in the topology after disconnecting and rejoining a cluster with the same node id (#1493)
• Ensure memory gets released on shut down, prevent crashes on instance shutdown (#1555)
• Turn on MLX_METAL_FAST_SYNCH for Ring instances, speeding up generation for large prompts with Ring instances (#1594)
• Prevent runners silently dying and detect GPU timeouts and out of memory errors (#1592)

Security

• Make trust_remote_code opt-in (default false) for custom models loaded from Huggingface (#1603 h/t @kevthehermit)

Full Changelog: https://github.com/exo-explore/exo/compare/v1.0.67...v1.0.68

EXO v1.0.67 Release Notes

This release adds support for Kimi-K2.5 and tensor sharding support for MiniMax M2.1.

Models

• Add support for Kimi K2.5 (#1302)
• Add tensor sharding support for MiniMax M2.1 (#1299)

Full Changelog: https://github.com/exo-explore/exo/compare/v1.0.66...v1.0.67

EXO v1.0.66 Release Notes

This is a stability release that fixes a regression with RDMA / Tensor Parallelism where models were getting stuck in LOADING state. It also fixes a download edge case with GLM 4.7 Flash and nodes getting stuck in UNKNOWN state / zombie states after periods of inactivity.

All models have been confirmed working with RDMA / Tensor Parallel on various configurations (including Mac Minis, MacBooks and Mac Studios). Thank you to users who reported bugs to help us resolve these issues - it helps a lot!

Bug Fixes

• Use EXO shard instead of upstream shard for all models, loading models layer-by-layer, fixing models getting stuck in LOADING state e.g.GLM-4.7-Flash-4bit, gpt-oss-120b-MXFP4-Q8 and Qwen3-Coder-480B-A35B-Instruct-8bit. Also fixes memory not being released when an instance is deleted. (#1291)
• Fix downloads getting stuck when model files change in Huggingface repo e.g. GLM-4.7-Flash-4bit which was updated upstream on Jan 25 (#1290)
• Always publish info gatherer events, preventing nodes getting stuck in UNKNOWN state / zombie states after a period of inactivity (#1283)
• Fix tool calls with empty text content (#1292)

MLX

• Upgrade mlx-lm to 0.30.5

Full Changelog: https://github.com/exo-explore/exo/compare/v1.0.65...v1.0.66

EXO v1.0.65 Release Notes

This release ships with stability fixes for RDMA and long running clusters (fixing the FAILED -> PREPARING loops for some clusters), as well as QoL features for managing downloads and a fix for GPT-OSS tool calling.

UX

• Add download and delete buttons to downloads UI (#1236, (#1237))

Bug Fixes

• Fix parsing logic for GPT-OSS tool calling (#1271)
• Fix Thunderbolt bridge cycle detection to include 2-node cycles (#1261)
• Fix placement filter to use subset matching instead of exact match (#1265)
• Fix instance port assignment, improving stability for RDMA clusters (#1268)
• Deprioritise uncertain ethernet devices, improving stability for RDMA clusters (#1267)
• Restore Thunderbolt Bridge LaunchDaemon, since TB bridge gets enabled by macOS on reboot automatically (#1270)
• Add back the EXO network profile creation to the LaunchDaemon (#1277)

Full Changelog: https://github.com/exo-explore/exo/compare/v1.0.64...v1.0.65

EXO v1.0.64 Release Notes

This release comes with support for GLM-4.7-Flash, IP-less RDMA discovery (removing the need for custom network locations) and OpenAI-compatible tool calling via the API. It also includes bug fixes for auto parallelism, fixing various models including Qwen, GPT-OSS and MiniMax that were getting stuck in LOADING / WARMING UP, as well as better error messages when things go wrong.

Model Support

• Added support for GLM-4.7-Flash (#1214)

API

• Added tool calling support to the OpenAI-compatible chat completions API (#1233)

UX

• Add proxy and custom SSL certificate support for corporate networks (#1189)
• More responsive node info by splitting information sources and state (#928, #1209)
• Less spammy logs (#1218, #1225)
• Better error messages when things go wrong (#1198, #1173, #1177)
• Detect Thunderbolt Bridge cycles and show a native prompt to disable Thunderbolt Bridge (#1222)
• Add the ability to filter nodes for placement in the dashboard by clicking them (#1248)

Bug Fixes

• Fix placement edge cases for heterogeneous devices with lopsided memory availabilities (#1200)
• Fix various issues with auto parallel for loading and sharding models (#1202, #1201, #1206, #1211, #1229, #1223)
• Prepend think tags for certain models that include a think tag in their chat template e.g. GLM 4.7 (#1186)

Full Changelog: https://github.com/exo-explore/exo/compare/v1.0.63...v1.0.64

EXO v1.0.63 Release Notes

This release comes with stability improvements for long-running clusters, and support for GLM 4.7 and MiniMax M2.1.

Model Support

• Added support for GLM 4.7 (#1147)
• Added support for MiniMax M2.1 (#1147)
• Tensor sharding for GPT-OSS (#1144)

UI

• Uninstall button in macOS app (#1077)

Bug Fixes

• Fix issues with nodes incorrectly dropping out of topology (#1164, #1170)
• Fix exo-bench for transformers 5.x (#1168)
• Fix gibberish responses for GPT-OSS (#1165)
• Increase rlimit to avoid hitting resource limit errors when launching more than 10 instances (#1148)
• Reduce false-positives on local network access warning in macOS app (#1136)
• Use user-provided seed for sampling in generation, enabling deterministic responses (#1094)
• Change status shown in Dashboard/macOS app from UNKNOWN to PREPARING (#1112)
• Cleaner scrolling in instances box on Dashboard (#1113)

Full Changelog: https://github.com/exo-explore/exo/compare/v1.0.62...v1.0.63