AnythingLLM v1.16.1

v1.16.1Desktop

AnythingLLM v1.16.1

Added 6
  • Navigation warning when canceling inference or leaving a page to prevent accidental abort of responses
  • Foundry Local embedded inference engine support on Windows 11 (version 24H2, build 26100 or later) using WinML
  • AWS Bedrock cross-region inference profile support with geo-prefixed profile IDs and manual region entry
  • LocalAI added as an image generation provider
  • LocalAI context window auto-detection from server config with manual override option
  • Silent and headless install and uninstall flags for Windows installer
Changed 8
  • Overhauled UI for long-horizon agentic tasks to roll up activity and chain of thought into a single clean, collapsible UI element
  • Replaced previous Snapdragon NPU engine with GenieX for improved performance on X and X2 Elite devices
  • Chain of thought and agent activity displayed as single collapsible component with live thought shown in header while streaming
  • Chain of thought rollup mirrored in the Assistant Panel
  • Foundry Local no longer prompts for setup when using built-in runtime and auto-loads models by capability detection
  • Onboarding light mode styling and redesigned LLM selection UI
  • Beacon now renders thinking blocks and markdown formatting correctly while streaming with collapsible thinking
  • Prompt input drafts now persist between navigation
Fixed 4
  • Fixed Gemini crash caused by content-header length mismatch on unpinned undici versions
  • Fixed crash when aborting Anthropic or Bedrock Anthropic response mid-stream in chat mode and sending follow-up
  • Fixed memory leak from leaked abort-stream listeners that caused pausing a response to wipe entire chat
  • Fixed crash when creating a new workspace

From AnythingLLM

Activity and Chain of Thought Overhaul

We overhauled our UI for long-horizon agentic tasks that show multiple thinking and tool calls to now roll up into a single clean, collapsible UI element.

This gives you more screen real estate for the actual response while still keeping the important call stack and thoughts the model produces in an easy-to-read format.

Navigation Warning

Recently, we added true abort signals across every supported LLM so that when you cancel inference or leave the page, we stop inference to save you on compute costs.

However, this had the unintended side effect of canceling inference when navigating away from a chat, preventing background completion. Now we warn you before aborting the response.

Roadmap item: We intend to make this user configurable so you can customize this behavior and allow background inference for those with more powerful or high-concurrency setups where multiple parallel inference sessions are feasible.

Foundry Local Embedded (Desktop Only)

FoundryLocal is an inference engine built by Microsoft that is pre-installed and available on Windows 11 (version 24H2, build 26100 or later).

This uses WinML to load models for your CPU, GPU, or NPU across any Windows hardware configuration. It's a great option for hardware configurations we don't have explicit support for. Keep in mind that model availability may be limited as FoundryLocal is still rolling out. Vision models are not available currently.

We partner with Microsoft for this, if you have bugs or issues we can help forward them to the FoundryLocal team

GenieX for Snapdragon (Desktop Only)

[!NOTE] This replaces our previous Snapdragon NPU engine. Any previous models from the old engine are deleted on update, and you'll need to re-download these optimized models. You should see a significant performance increase for NPU models and regular GGUFs.

We highly recommend using only GGUF models for this engine. Qualcomm NPU-only models have 4K context windows and will result in a poor agent experience.

GenieX is an open-source inference engine built by Qualcomm (previously NexaAI) that efficiently runs GGUFs and Qualcomm AI Hub models on NPU for X and X2 Elite devices.

AnythingLLM offers fully managed and built-in support by selecting the "AnythingLLM GenieX" provider in the dropdown. Both GGUFs and NPU-only models can leverage the NPU, giving you the power efficiency and intelligence of both.

We partner with Qualcomm for this, if you have bugs or issues we can help forward them to the GenieX team or you can make an issue on their GitHub. s/o @zhiyuan8, @alexchen4ai & @alanzhuly

New Features
  • Native Foundry Local SDK on Windows (x64/ARM64) - the Foundry provider is now fully self-contained and no longer requires a separate Foundry Local install, with GPU/NPU-optimized model variants exposed in the catalog
  • GenieX runtime support for Windows ARM64 devices
  • AWS Bedrock cross-region inference profile support - geo-prefixed profile IDs (us., eu., global.) now appear in the model dropdown and route correctly, plus manual region entry for regions outside the preset list (GovCloud, specialized partitions)
  • LocalAI added as an image generation provider, listing only image-capable models from your server
  • LocalAI context window auto-detection - context windows are read per-model from your server's config, with the manual setting now an optional override
  • Silent/headless install and uninstall flags for the Windows installer - see the Windows installation docs
Improvements
  • Chain of thought and agent activity are now a single collapsible component - one line when collapsed, one step per thought with a connected rail when expanded, and the live thought shown in the header while streaming
  • The new chain of thought rollup is now mirrored in the Assistant Panel
  • Leaving a thread or navigating away mid-inference now warns you before the response is lost, instead of silently killing the generation
  • Foundry Local no longer prompts for setup when using the built-in runtime, and auto-loads models by capability detection
  • Onboarding light mode styling and a redesigned LLM selection UI
  • Beacon now renders thinking blocks and markdown formatting correctly while streaming, with collapsible thinking and a fix for model detail text overflow
  • Windows installer now includes a sidebar panel image
  • Prompt input drafts now persist between navigation
Bug Fixes
  • Fixed a Gemini crash caused by a content-header length mismatch on unpinned undici versions - now compatible with both v6 and v7
  • Fixed a crash when aborting an Anthropic (or Bedrock Anthropic) response mid-stream in chat mode and then sending a follow-up
  • Fixed a memory leak from leaked abort-stream listeners, which also caused pausing a response to wipe the entire chat
  • Fixed a crash when creating a new workspace
  • Fixed missing setup CTAs on modals introduced by the new uniform modal component - Scheduled Job skill setup on both the category and tool views, Community Hub connection key, and the experimental features reject button
  • Fixed the intermediate loader being too tall while waiting for an agent capability response
  • Fixed the stray border around the delete button in Workspace Chats
  • Fixed reasoning getting stuck on "thinking" when a model never closes its think tag
  • Fixed agent websocket sends firing while the connection was still opening
All Changes
New Contributors

Full Changelog: https://github.com/Mintplex-Labs/anything-llm/compare/v1.16.0...v1.16.1

View original

Upgraded? How did it go?

Discussion