What’s New

LocalAI

AI

LocalAI release notes.

Latest v4.7.1 · by LocalAIWebsitemudler/LocalAI

Changelog

v4.7.0

Added
  • Managed voice cloning profiles with UI recording/uploading, saving reusable profiles, and referencing via localai://voice-profiles/<id> URI
  • REST endpoints for voice profile management: GET/POST /api/voice-profiles, GET /api/voice-profiles/:id/audio, DELETE /api/voice-profiles/:id
  • MCP admin tools for voice profiles: list_voice_profiles, create_voice_profile, delete_voice_profile
  • LongCat video backend for text-to-video, image-to-video, and audio-driven talking-avatar generation
  • Support for interleaved reasoning and tool calls in assistant turns with reasoning_content alias and Anthropic thinking block support
  • moss-transcribe-cpp backend for one-pass diarized transcription with multi-speaker detection and timestamps
  • Real streaming TTS support in vibevoice-cpp with true incremental audio streaming
  • F5-TTS integration for voice cloning in CrispASR
  • context_size: -1 to run models at their full trained context window using GGUF metadata
  • GPU device selection for llama.cpp via options [devices:CUDA1,CUDA2] to restrict offload to named GPUs
  • DFlash speculative-decoding gallery models for Qwen 4B, 9B, 27B, and 35B-A3B
  • known_input_modalities and known_output_modalities configuration fields
Changed
  • Voice cloning capability auto-discovered across 12+ TTS backends without hardcoded backend names
  • Transcription language and translate settings now reach the backend
Fixed
  • Model-load cooldown prevents deterministically-failing models from respawning on every poll and leaking VRAM
  • OIDC support for EC/PS/EdDSA-signed tokens

🎉 LocalAI 4.7.0 Release! 🚀

LocalAI 4.7.0 is out!

This release widens what LocalAI can generate and how you drive it: a UI-managed voice cloning library, local video and audio-driven avatar generation, and interleaved reasoning that travels with tool calls. It also lands new audio engines (one-pass diarized transcription, F5-TTS, true streaming TTS) and a batch of reliability fixes across auth, transcription, and the model gallery.

Highlights:

  • 🎙️ Managed voice cloning - record or upload a consented reference in the UI, save it as a reusable profile, and reference it from any cloning-capable TTS backend with a stable localai://voice-profiles/<id> URI. No more hand-edited YAML or copying audio into model folders.
  • 🎬 Local video & avatars - a new longcat-video backend brings text-to-video, image-to-video, and audio-driven talking-avatar generation, wired into the Studio UI with audio and reference-image controls.
  • 🧠 Interleaved thinking with tool calls - an assistant turn can now carry reasoning and tool_calls together and keep the reasoning across the tool-result loop, with a reasoning_content inbound alias and Anthropic thinking block support.
  • 🗣️ New audio engines - one-pass diarized transcription (moss-transcribe-cpp), F5-TTS voice cloning in CrispASR, and true streaming TTS in vibevoice-cpp (time-to-first-audio 2.38s vs 39.96s on CPU).
  • 🎚️ Auto full context - context_size: -1 runs a model at its full trained context window, read from GGUF metadata per-model, with a VRAM-fit warning.
  • 🖥️ Sharper control - pick which GPUs llama.cpp offloads to (devices:), and a new model-load cooldown stops a deterministically-failing model from respawning its backend on every poll and leaking VRAM.

Plus DFlash speculative-decoding gallery models, an OIDC fix for EC/PS/EdDSA-signed tokens, transcription language/translate settings that reach the backend, and the usual set of dependency updates.

📌 TL;DR
AreaSummary
🎙️ Voice LibraryAdmin-managed voice cloning profiles: record/upload consented reference audio in the UI, preview, and reference via localai://voice-profiles/<id>. New GET/POST/DELETE /api/voice-profiles, MCP tools (list/create/delete_voice_profile), and a typed tts.voice_cloning config. Cloning declared as a capability across 12+ TTS backends (self-discovered, no hardcoded backend names).
🎬 LongCat video & avatarsNew longcat-video Python backend: text-to-video, image-to-video, and LongCat-Video-Avatar 1.5 audio-driven avatars. Gallery entries longcat-video and longcat-video-avatar-1.5; new known_input_modalities/known_output_modalities config fields; /video endpoint extended with staged audio. CUDA 12/13 x86_64 + CUDA 13 ARM64 images.
🧠 Interleaved thinkingAssistant turns carry reasoning + tool_calls together and preserve reasoning across the tool loop. reasoning_content accepted as an inbound alias; Anthropic Messages local path round-trips thinking blocks (gated on thinking: {type: "enabled"}).
🗣️ moss-transcribe-cppNew Go backend over the C++/ggml MOSS-Transcribe-Diarize port: joint multi-speaker transcription + diarization + timestamps in one pass. Gallery model moss-transcribe-cpp-0.9b. Offline; 1.6-2.2x faster than reference on CPU, bit-exact on CUDA.
⚡ vibevoice streaming TTSReal incremental streaming via vv_capi_tts_stream: TTFA 2.38s vs 39.96s batch (~17x) on CPU for VibeVoice-Realtime-0.5B.
🎨 F5-TTSF5-TTS linked into CrispASR + f5-tts-crispasr gallery model (24kHz, voice cloning via voice:/voice_text: options).
🎚️ context_size: -1Negative context_size (or LOCALAI_CONTEXT_SIZE=-1) resolves to the model's n_ctx_train from GGUF metadata, with a GPU-only VRAM-fit warning and a safe clamp so no backend ever sees a negative window.
🖥️ llama.cpp device selectionoptions: [devices:CUDA1,CUDA2] restricts offload to named GPUs (from --list-devices).
🛡️ Model-load cooldownA failed load enters a cooldown (default 10s, geometric growth capped at 5m) so polling clients get 503 + Retry-After instead of respawning a crashing backend and leaking GPU memory.
🧠 Speculative decoding modelsFour Qwen DFlash gallery entries (4B / 9B / 27B / 35B-A3B), each bundling target + drafter, spec_type:draft-dflash.
🚀 New Features & Major Enhancements
🎙️ Managed voice cloning profiles (Voice Library)

Voice cloning becomes a first-class, UI-driven workflow. Record or upload consented reference audio in the React UI, normalize it to WAV, preview it, and save it as a named profile. Reference the profile from the TTS UI or /v1/audio/speech with a stable localai://voice-profiles/<id> URI, no YAML editing and no copying audio into model directories.

  • New REST surface: GET /api/voice-profiles, GET /api/voice-profiles/:id/audio, POST /api/voice-profiles (admin), DELETE /api/voice-profiles/:id (admin), surfaced in Swagger, /api/instructions, and the auth capability registry. MCP admin tools list_voice_profiles, create_voice_profile, delete_voice_profile.
  • New typed config tts.voice_cloning (bool) opts custom model names in or out of Voice Library compatibility; false also rejects saved profile references with HTTP 400. Request precedence is voice -> tts.voice -> tts.audio_path; existing options still work with no breaking change.
  • Compatibility is server-discovered from backend capabilities (the frontend hardcodes no backend names) and offers gallery models to install when none are present. Cloning is declared in the capability registry for vllm-omni, vibevoice-cpp, coqui, pocket-tts, qwen-tts, qwen3-tts-cpp, faster-qwen3-tts, fish-speech, neutts, chatterbox, voxcpm, omnivoice-cpp, and model-dependent crispasr.
  • Cross-backend contract: reference WAV + exact transcript via params.ref_text. E2E verified on qwen3-tts-cpp (4.47s reference produced a non-silent 3.44s 24kHz WAV).

🔗 PRs: #10799

🎬 LongCat video and avatar generation

A new longcat-video Python backend brings local video generation to LocalAI: text-to-video, image-to-video, and LongCat-Video-Avatar 1.5 for audio-driven talking avatars. It is wired into the React Studio UI with audio and reference-image controls.

  • Gallery entries longcat-video (text+image input, video output) and longcat-video-avatar-1.5 (text+image+audio input, video output). Backend images: CUDA 12 and CUDA 13 x86_64, plus CUDA 13 ARM64 (DGX Spark / NVIDIA ARM64 guidance included).
  • Introduces declarative capability metadata: new known_input_modalities / known_output_modalities config fields so generic code discovers what a checkpoint accepts instead of branching on backend or checkpoint names. The HF importer emits matching self-describing recipe metadata.
  • The /video endpoint gains staged audio and video-generation parameters, with distributed audio staging (input bounded to 128 MiB). Avatar mode supports multi-segment generation, BF16 or optional INT8 quantization, and distillation settings, documented in a new features/longcat-video.md page.

🔗 PRs: #10792

🧠 Interleaved thinking with tool calls

An assistant turn can now carry reasoning and tool_calls together, and the reasoning survives the tool-result loop across turns. This is uniform, tested, and documented behavior rather than a per-backend accident.

  • OpenAI chat messages accept reasoning_content as an inbound alias for the canonical reasoning field (vLLM/DeepSeek/cogito-style clients emit it); emission is unchanged and the canonical field wins when both are present.
  • The Anthropic Messages local path round-trips thinking blocks: inbound thinking blocks parse into the message reasoning, and a thinking block is emitted before tool_use on both non-streaming and streaming responses, gated on the request param thinking: {type: "enabled"}. The cloud-proxy passthrough path is untouched.
  • Verified live: lfm2.5-8b-a1b and gemma-4-e2b return reasoning plus structured tool_calls in a single turn. A new features/interleaved-thinking.md doc is cross-linked from the model-configuration, text-generation, and functions guides.

🔗 PRs: #10744

🗣️ New audio engines: diarized transcription, F5-TTS, and streaming TTS

Three additions widen the audio surface:

  • moss-transcribe-cpp (#10756): a new Go backend that dlopens the C++/ggml MOSS-Transcribe-Diarize port to do joint multi-speaker transcription, diarization, and timestamps in a single offline pass. Gallery model moss-transcribe-cpp-0.9b (default q5_k GGUF). The ggml port is byte-exact to the reference PyTorch and 1.6-2.2x faster on CPU, bit-exact on CUDA (verified on Blackwell / Jetson Thor). Builds across the full Linux matrix plus Darwin/Metal.
  • F5-TTS in CrispASR (#10753): links the F5-TTS static runtime into the CrispASR build (SWivid, MIT; a 22-layer DiT flow-matching model with a built-in Vocos vocoder) and ships an f5-tts-crispasr gallery model. Produces 24kHz mono audio and auto-detects as f5-tts (no backend: selector). Voice cloning via options: [voice:/path/ref.wav, voice_text:Transcript...]; note F5-TTS runs a 32-step ODE solver, so CPU synthesis is compute-heavy.
  • vibevoice-cpp true streaming (#10764): replaces whole-clip-then-chunk synthesis with real incremental streaming through the new vv_capi_tts_stream callback ABI. Time-to-first-audio drops from 39.96s (batch) to 2.38s (streaming), about 17x, on a CPU-only box for VibeVoice-Realtime-0.5B. Scope is the realtime-0.5B model; the non-streaming path is unchanged.

🔗 PRs: #10756, #10753, #10764

🎚️ Auto full context with context_size: -1

A context_size: -1 sentinel (any negative value) now makes a model run at its full trained context, resolved per-model from the GGUF n_ctx_train metadata at load. This lets you opt a model into its true maximum window even when a gallery YAML already pins a value, without hardcoding a number. The global equivalents LOCALAI_CONTEXT_SIZE=-1 / --context-size -1 make every model resolve to its own trained max, while an explicit per-model value still wins.

Three defense layers keep it safe: GGUF resolution degrades to the default with a warning if metadata lacks a usable max; a GPU-only warnIfContextExceedsVRAM helper logs (never blocks) when the window likely will not fit; and the backend options layer clamps any residual negative to the default so no backend receives a negative n_ctx. The previously-only path (unset context_size) is unchanged.

🔗 PRs: #10752

🖥️ GPU device selection and model-load cooldown
  • llama.cpp device selection (#10724): a new device / devices option in the llama.cpp options: array maps to upstream --device, so you can restrict offload to specific GPUs (for example excluding a display or debug GPU). Example: options: [devices:CUDA1,CUDA2,CUDA3]. Device names come from llama-server --list-devices.
  • Model-load failure cooldown (#10728): after a load fails, new independent load triggers are refused during a cooldown window, returning a typed error mapped to 503 + Retry-After instead of respawning the backend on every poll. This stops a deterministically-failing model from leaking GPU/CUDA state (a reporter saw leaked contexts climb to ~58 GB) and, under LOCALAI_SINGLE_ACTIVE_BACKEND, from stealing the active slot from healthy models. Configurable via --model-load-failure-cooldown / LOCALAI_MODEL_LOAD_FAILURE_COOLDOWN (default 10s, 0 disables); the cooldown doubles per consecutive failure, caps at 5m, and resets on a successful load. Coalesced followers of a genuine concurrent burst still get their one retry.

🔗 PRs: #10724, #10728

🧠 Models
  • Qwen DFlash speculative-decoding models (#10791): four gallery entries for the llama-cpp backend, each bundling a full target model plus its small block-diffusion drafter (drafters are not standalone chat models): qwen3-4b-dflash, qwen3.5-9b-dflash, qwen3.6-27b-dflash, qwen3.6-35b-a3b-dflash. Per-entry config is flash_attention: on, draft_model:, use_jinja:true, spec_type:draft-dflash, spec_n_max:15. Requires the pinned llama.cpp with upstream DFlash support; every bundled drafter is verified general.architecture = dflash (fork-only dflash-draft GGUFs are intentionally excluded because they fail to load). GPU recommended.
  • Inference defaults (#10741): adds recommended sampling defaults for deepseek-v4 (auto-generated from unsloth), so LocalAI applies correct generation parameters for the family automatically.
  • Plus new gallery models added via the gallery agent (#10743, #10755).
🐛 Bug Fixes (recap)
  • fix(auth): accept EC/PS/EdDSA-signed OIDC ID tokens, not just RS256 (OIDC login was 500ing at callback with EC-signed tokens, e.g. Authentik) - #10736
  • fix(transcription): honor model-config parameters.language/parameters.translate and the OpenAI language form field, which were silently ignored for multipart uploads - #10731
  • fix(backends): opus and local-store now refuse foreign model loads, so an LLM with no explicit backend cannot silently bind to the audio codec or vector store during backend probing - #10769
  • fix(gallery): backend (re)install is now a clean atomic replace (stage, validate, swap, rollback) instead of an overlay, so stale files from a prior version no longer shadow the new one - #10726
  • fix(vram): report the largest single GGUF quant instead of summing every quant in an HF repo, so a 9B model no longer shows as 71 GB / "May not fit" - #10707
  • fix(logs): capture backend stdout/stderr into the log store by default in single mode, so the Backend Logs page is populated out of the box - #10742
  • fix(vllm): pin the L4T arm64 backend to vllm==0.24.0 for GB10 / DGX Spark stability (0.23 crashes deterministically on cold loads and pins GPU memory) - #10725
  • fix(ds4): bundle the full transitive runtime dependency closure so the from-scratch DS4 image no longer exits 127 on a missing gRPC library - #10783
  • fix(diffusers,vllm-omni,tinygrad): save generated images as PNG explicitly, fixing an unknown file extension: .tmp crash right after successful inference - #10729
  • fix(react-ui): preserve uploaded file content when regenerating a non-last answer (attachments were silently dropped after forking and regenerating a file turn) - #10819
  • fix(ui): prevent a large data table from breaking the flexbox layout and forcing a page-wide horizontal scrollbar - #10754
👒 Dependencies

Submodule and backend bumps this cycle:

  • ggml-org/llama.cpp x7
  • CrispStrobe/CrispASR x7
  • vllm-metal (darwin) x6
  • ServeurpersoCom/qwentts.cpp x4
  • ServeurpersoCom/omnivoice.cpp x4
  • leejet/stable-diffusion.cpp x3
  • ikawrakow/ik_llama.cpp x3
  • ggml-org/whisper.cpp x2
  • mudler/moss-transcribe.cpp x1
  • mudler/locate-anything.cpp x1
  • vllm-project/vllm 0.24.0 -> 0.25.0 (Python backend) and cu130 wheel to 0.25.0

Plus grpcio 1.81.1 -> 1.82.1, charset-normalizer >=3.4.9, and GitHub Actions bumps (actions/cache 4 -> 6, actions/stale 10.3.0 -> 10.4.0).

📖 Documentation
  • Refreshed the LocalAI homepage to frame the project as a modular multimodal AI runtime: breadth lanes (reason, listen and speak, create, see, act), the small-core plus on-demand-backends architecture, the native engines built by the LocalAI team, and the scale path from laptop to team server to cluster - #10780
  • Docs-site version bump - #10709
🙌 New Contributors
  • @rvmz made their first contribution in #10724
  • @hogeheer499-commits made their first contribution in #10783
  • @ajuijas made their first contribution in #10819

Full Changelog: https://github.com/mudler/LocalAI/compare/v4.6.2...v4.7.0

View originalPermalink

v4.6.2

Added
  • Add MiniCPM series models to model gallery
Changed
  • Use slices.Contains to simplify code
Fixed
  • Shard single-arch backend matrix under GitHub's 256-job limit
What's Changed
👒 Dependencies
Other Changes
New Contributors

Full Changelog: https://github.com/mudler/LocalAI/compare/v4.6.1...v4.6.2

View originalPermalink

v4.6.1

Added
  • Add GET /v1/models/capabilities endpoint
  • Add native Prometheus metrics for agent chat runs
Fixed
  • Log the real cause of OIDC/OAuth user-info failures
  • Center the home empty-state wizard in UI
  • Don't persist request-scoped reasoning_effort as an operator disable
  • Scope generated-content and upload dirs to the current user on startup
  • Cap auto-derived context to fit VRAM
  • Cap single-pass embedding batch to fit VRAM
What's Changed
Other Changes

Full Changelog: https://github.com/mudler/LocalAI/compare/v4.6.0...v4.6.1

View originalPermalink

v4.6.0

Added
  • Realtime sessions eagerly warm the entire pipeline (VAD, ASR, LLM, TTS) at session start
  • New POST /backend/load and POST /v1/backend/load API endpoints to load models into memory
  • Load into memory UI button in the built-in chat interface
  • Chat forking capability allowing regeneration of any assistant answer, branching from any turn, duplicating conversations, and copying as Markdown
  • Prometheus counter localai_pii_events_total for tracking PII detections, masks, and blocks
Changed
  • POST /backends/apply and LOCALAI_EXTERNAL_BACKENDS no longer re-pull already-installed backends unless force: true is specified
Fixed
  • AMD ROCm ggml audio backends now compile with -DGGML_HIP=ON and link HIP for real GPU offload
  • hipBLASLt kernel-tuning data (TensileLibrary files) is now bundled and HIPBLASLT_TENSILE_LIBPATH is exported
  • rocm-vllm backend now installs from the AMD ROCm wheel index instead of the CUDA-only PyPI wheel
  • ASIC ID table (amdgpu.ids) is now symlinked so compute-only hipblas images can find it
  • Dead worker no longer pins the per-model PostgreSQL advisory lock during distributed model loads
  • Backend workers now self-terminate on parent death instead of holding VRAM
  • Gallery config URL fetches are now validated against private, loopback, link-local, and cloud-metadata addresses to prevent SSRF
  • Non-streaming vLLM tool calls are now restored
  • MLX and Python backends now decode tool-call arguments for chat templates and split closing-only reasoning blocks

🎉 LocalAI 4.6.0 Release! 🚀

LocalAI 4.6.0 is out!

This is a reliability-focused release: AMD ROCm backends now run on-GPU at full speed, distributed model loads no longer wedge when a worker dies, and realtime sessions warm up predictably. It also brings conversation forking to the built-in chat UI, a Prometheus counter for PII/audit events, and an SSRF fix for the model gallery.

Highlights:

  • 🔴 AMD ROCm runs correctly - ggml audio backends offload to the GPU, hipBLASLt kernel-tuning data is bundled (no more slow generic kernels), rocm-vllm installs the right wheel, and the ASIC ID table is found.
  • 🎙️ Predictable realtime - sessions eagerly warm the whole pipeline (VAD, ASR, LLM, TTS) up front, so the first turn no longer pays per-model cold-start stalls, plus a new POST /backend/load API and "Load into memory" UI button.
  • 🌿 Forking chat - retry any assistant answer, branch a new chat from any point, duplicate, or copy the whole conversation, directly in the built-in UI.
  • 🛡️ Distributed hardening - a dead worker can no longer pin the model-load advisory lock (the ~15-minute wedge is gone), and orphaned backend workers self-terminate instead of holding VRAM.
  • 📊 PII/audit metrics - PII detections/masks/blocks are exported as a Prometheus counter, so you can alert when the filter stops firing.
  • 🔒 Gallery SSRF fix - POST /models/apply config-URL fetches are validated against private/loopback/metadata addresses.

Plus idempotent backend installs, tool-calling and reasoning fixes across the vLLM and Python/MLX backends, cloud-proxy compatibility with the newest reasoning models, and the usual set of dependency updates.

📌 TL;DR
AreaSummary
🔴 AMD ROCm reliabilityggml audio backends now compile with -DGGML_HIP=ON and link HIP (real GPU offload); hipBLASLt TensileLibrary data bundled + HIPBLASLT_TENSILE_LIBPATH exported; rocm-vllm installs from the AMD wheel index on Python 3.12; amdgpu.ids symlinked so the ASIC table is found.
🎙️ Realtime warm-up + load APISessions block-warm the full pipeline at start (errors surface up front); new POST /backend/load / POST /v1/backend/load, a "Load into memory" UI action, and a load_model MCP tool. Opt out per pipeline with disable_warmup: true.
🌿 Forking chatRegenerate any assistant answer (not just the last), branch a new chat from any turn, duplicate a chat, or copy it as Markdown - all client-side in the React UI.
🛡️ Process & distributed lifecycleA dead worker no longer pins the per-model PostgreSQL advisory lock (bounded load ceiling + context-scoped lock_timeout); backend workers self-terminate on parent death (LOCALAI_BACKEND_PARENT_WATCH); the watchdog stops logging optional Free() as an error.
⚙️ Idempotent backend installsPOST /backends/apply and the LOCALAI_EXTERNAL_BACKENDS boot loop no longer re-pull an already-installed backend unless force: true.
📊 PII/audit Prometheus counterlocalai_pii_events_total{kind,origin,action,direction} on /metrics, complementing the /api/pii/events ring buffer.
🔒 Gallery SSRF hardeningGallery config URL fetches run through ValidateExternalURL, blocking private, loopback, link-local, and cloud-metadata addresses.
🧩 Tool-calling & reasoning fixesNon-streaming vLLM tool calls restored; MLX/Python backends decode tool-call arguments for chat templates and split closing-only </think> reasoning blocks.
🚀 New Features & Major Enhancements
🔴 AMD ROCm backends run correctly on-GPU

Four coupled fixes make ROCm/hipBLAS backends actually run on AMD hardware, and at full speed, instead of silently falling back to CPU or slow generic kernels:

  • GPU offload for ggml audio backends (#10667): rocm-qwen3-tts-cpp, rocm-omnivoice-cpp, acestep-cpp, and vibevoice-cpp were building CPU-only because their Makefiles passed the no-op -DGGML_HIPBLAS=ON (upstream ggml only understands -DGGML_HIP=ON) and the CMake link loop omitted hip. They now use the same hipblas recipe as llama-cpp and link the HIP backend.
  • hipBLASLt kernel-tuning data (#10660, #10672): the packager bundled rocBLAS data but not the parallel hipBLASLt TensileLibrary_lazy_gfx*.dat files, so every arch silently used slow kernels and logged Cannot read "TensileLibrary_lazy_gfx*.dat". The data is now bundled and HIPBLASLT_TENSILE_LIBPATH is exported by the llama-cpp and turboquant run.sh.
  • rocm-vllm installs the right wheel (#10642, #10651): the backend was pulling the CUDA-only PyPI vllm (fatal ModuleNotFoundError: No module named 'vllm' on AMD). It now pins CPython 3.12 and installs vLLM from the ROCm wheel index (https://wheels.vllm.ai/rocm/).
  • ASIC ID table found (#10624, #10627): the compute-only hipblas image lacks /opt/amdgpu/share/libdrm/amdgpu.ids, so every model load warned. Ubuntu's libdrm-common copy is now symlinked into place.

🔗 PRs: #10667, #10672, #10651, #10627

🎙️ Realtime: eager pipeline warm-up + a load-into-memory API

Realtime voice sessions now eagerly and blockingly warm the entire pipeline (VAD, transcription, LLM, TTS, sound detection, voice recognition) at session start instead of lazy-loading each sub-model on first use. The first turn no longer pays per-model cold-start stalls, and model-load errors surface up front at session start (as model_load_error) rather than mid-stream. Pipeline sub-models load concurrently, so a session warms in the time of its slowest stage, not the sum, and a failed stage names every broken model in a joined error.

This also adds a LocalAI-native POST /backend/load (and /v1/backend/load), the inverse of /backend/shutdown, exposed as a "Load into memory" UI action and a load_model MCP admin tool, so admins can pre-warm any model (including full pipelines) on demand. The --load-to-memory startup flag now routes through the same engine. Opt out per pipeline with disable_warmup: true.

🔗 PRs: #10662

🌿 Forking chat in the built-in UI

The React chat UI gains conversation-management tools: regenerate any assistant answer (not just the last), branch a new chat from any answer, duplicate a chat into an independent copy, or copy the whole conversation to the clipboard as Markdown. Retrying a mid-conversation answer correctly truncates the conversation before re-asking, both in the DOM and in the request payload (this also fixes a latent stale-closure bug where a mid-conversation retry sent the downstream turns back to the model). All client-side, no backend changes.

🔗 PRs: #10654

🛡️ Sturdier process and distributed lifecycle
  • Dead-worker advisory-lock wedge (#10600): a distributed worker going mid-load could pin a per-model PostgreSQL advisory lock and fail every subsequent request to that model with 55P03 for ~15 minutes. The detached load context is now bounded by a model-load ceiling, the install wait honors cancellation via singleflight.DoChan, and lock_timeout is scoped to the caller's context budget instead of a deployment-global GUC.
  • Parent-death safety net (#10639): if LocalAI is SIGKILLed before teardown, spawned backend workers used to get reparented to init and linger, holding VRAM and their port. Each backend now polls its parent PID and self-terminates on reparenting. Configurable via LOCALAI_BACKEND_PARENT_WATCH (default on, auto-off on Windows) and LOCALAI_BACKEND_PARENT_WATCH_INTERVAL (default 2s). C++ coverage is llama-cpp for now; Python covers all backends.
  • Quieter watchdog (#10602, #10607): the optional Free() RPC returns gRPC Unimplemented for many backends and the federation proxy, so the watchdog no longer logs a misleading Error freeing GPU resources on eviction. A new grpcerrors.IsUnimplemented helper distinguishes it from genuine failures.
  • Idempotent backend installs (#10643): POST /backends/apply and the LOCALAI_EXTERNAL_BACKENDS boot loop no longer re-download and re-extract an already-installed backend on every apply/boot. Pass "force": true (the UI's install button still does, doubling as "Reinstall").

🔗 PRs: #10600, #10639, #10607, #10643

📊 PII/audit events as a Prometheus counter

The PII middleware / MITM audit pipeline now emits a single monotonic counter, localai_pii_events_total{kind, origin, action, direction}, on /metrics, instrumented at the EventStore.Record choke point. Labels are cardinality-bounded (no pattern or user IDs). This complements the capacity-bound /api/pii/events ring buffer and, crucially, makes silent filter failure alertable: rate() on the counter detects that the PII filter stopped firing after a deploy.

🔗 PRs: #10641

🔒 Gallery SSRF hardening

POST /models/apply with an empty id fetches the supplied url directly; in a default Docker setup (no API key) any reachable client could probe internal services or cloud-metadata (169.254.169.254) and exfiltrate a slice via the job error. Gallery config fetches now run through the existing ValidateExternalURL guard (the same one protecting the CORS proxy and media downloads), blocking private, loopback, link-local, unspecified, and metadata addresses. Only plain http(s):// is validated; huggingface://, github:, oci://, ollama://, and file:// are untouched.

🔗 PRs: #10673

🐛 Bug Fixes (recap)
  • fix(vllm): restore non-streaming tool-call extraction that regressed after #10351 (a capability flag was mistaken for run state) - #10638
  • fix(python-backends): decode tool-call arguments for chat templates (unbreaks MLX/Qwen3.5 agent loops) and split reasoning when a model emits only a closing </think> - #10658
  • fix(cloud-proxy): drop temperature/top_p and send max_completion_tokens so routing to the newest reasoning models (Claude Opus 4.x, GPT-5.x) stops 400ing - #10640
  • fix(config): revert defaulting swa_full:true for sliding-window-attention models (restores the memory-light reduced KV cache; still available as an explicit per-model opt-in) - #10674
  • fix(kokoros): implement the AudioTranscriptionLive trait stub so the backend compiles against the updated proto - #10612
  • fix(launcher): keep the desktop launcher's data/config under ~/.localai instead of the GUI's working directory - #10610, #10613
👒 Dependencies

Submodule and backend bumps this cycle:

  • ggml-org/llama.cpp x4
  • ikawrakow/ik_llama.cpp x4
  • CrispStrobe/CrispASR x4
  • leejet/stable-diffusion.cpp x3
  • vllm-metal (darwin) x3
  • ggml-org/whisper.cpp x2
  • mudler/parakeet.cpp x1
  • localai-org/privacy-filter.cpp x1
  • vllm-project/vllm cu130 wheel to 0.24.0

Plus new gallery models added via the gallery agent (#10663, #10644).

📖 Documentation
  • Docs version bump for the release - #10614
🙌 New Contributors
  • @alaningtrump made their first contribution in #10657

Full Changelog: https://github.com/mudler/LocalAI/compare/v4.5.6...v4.6.0

View originalPermalink

v4.5.6

Added
  • SyncedMap component for distributed cross-replica state
  • LOCALAI_DISTRIBUTED_SHARED_MODELS environment variable to skip staging on shared volumes
  • Voice-detect and face-detect ggml backends to replace Python insightface/speaker-recognition
  • Semantic VAD EOU token for realtime
Changed
  • Migrate finetune/quant/agent-tasks to cross-replica state
Fixed
  • Missing agent NATS permission in distributed mode
  • Fall back to DefaultContextSize for unparseable GGUFs and pin NVFP4 gallery context_size
  • Gallery mmproj/model quant matching to prevent F16 from selecting BF16
  • Return empty backend list for agent nodes instead of failing backend.list
  • Bound resume-stream buffer and enforce response ownership in openresponses
  • IK-llama multimodal path ported to mtmd API
  • Kokoro add explicit click dependency so spacy CLI works on intel build
  • Launcher binary download/upgrade with resume, rate-limit, and improved UX
  • Fish-speech allow invalid_reference_casting so tokenizers builds on darwin
  • OCI retry layer downloads on transient network errors
  • SGLang parse tool_call function arguments before applying the chat template
  • OpenAI stop max_tokens streaming retry loop on reasoning models
  • Import derive model name from selected GGUF for repo-root URIs
  • Avoid quadratic-time debug logging in CleanupLLMResult and ParseFunctionCall
Removed
  • Drop the darwin/metal build target for fish-speech
What's Changed
👒 Dependencies
Other Changes
New Contributors

Full Changelog: https://github.com/mudler/LocalAI/compare/v4.5.5...v4.5.6

View originalPermalink

v4.5.5

Added
  • Add 1 new model to the model gallery
Fixed
  • Repair release CI build and test breaks for kokoros, fish-speech, llama-cpp-quantization, and sglang backends
  • Whisper Darwin run.sh loads whichever fallback library exists (.so or .dylib)
What's Changed
Other Changes

Full Changelog: https://github.com/mudler/LocalAI/compare/v4.5.4...v4.5.5

View originalPermalink

v4.5.3

Added
  • Sign and notarize the DMG, app, and server binary on macOS
Fixed
  • Set rpath on the piper darwin binary so it can load its bundled libs
  • Darwin packaging for silero-vad backend
  • Show a node's existing labels on the detail view
  • Bundle transitive dependencies of GPU runtime libs
  • Broadcast admin model-config changes across replicas in distributed mode
  • Stop reinterpreting plain-string message content as JSON in llama-cpp
What's Changed
Other Changes

Full Changelog: https://github.com/mudler/LocalAI/compare/v4.5.2...v4.5.3

View originalPermalink