v4.5.6
Added
- SyncedMap component for distributed cross-replica state
- LOCALAI_DISTRIBUTED_SHARED_MODELS environment variable to skip staging on shared volumes
- Voice-detect and face-detect ggml backends to replace Python insightface/speaker-recognition
- Semantic VAD EOU token for realtime
Changed
- Migrate finetune/quant/agent-tasks to cross-replica state
Fixed
- Missing agent NATS permission in distributed mode
- Fall back to DefaultContextSize for unparseable GGUFs and pin NVFP4 gallery context_size
- Gallery mmproj/model quant matching to prevent F16 from selecting BF16
- Return empty backend list for agent nodes instead of failing backend.list
- Bound resume-stream buffer and enforce response ownership in openresponses
- IK-llama multimodal path ported to mtmd API
- Kokoro add explicit click dependency so spacy CLI works on intel build
- Launcher binary download/upgrade with resume, rate-limit, and improved UX
- Fish-speech allow invalid_reference_casting so tokenizers builds on darwin
- OCI retry layer downloads on transient network errors
- SGLang parse tool_call function arguments before applying the chat template
- OpenAI stop max_tokens streaming retry loop on reasoning models
- Import derive model name from selected GGUF for repo-root URIs
- Avoid quadratic-time debug logging in CleanupLLMResult and ParseFunctionCall
Removed
- Drop the darwin/metal build target for fish-speech
What's Changed
👒 Dependencies
- chore(deps): bump actions/cache from 4 to 6 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/10593
Other Changes
- docs: :arrow_up: update docs version mudler/LocalAI by @localai-bot in https://github.com/mudler/LocalAI/pull/10560
- fix(distributed): missing agent NATS permission by @ALameLlama in https://github.com/mudler/LocalAI/pull/10549
- feat(distributed): SyncedMap component + migrate finetune/quant/agent-tasks to cross-replica state by @localai-bot in https://github.com/mudler/LocalAI/pull/10542
- chore(fish-speech): drop the darwin/metal build target by @localai-bot in https://github.com/mudler/LocalAI/pull/10561
- fix(config): fall back to DefaultContextSize for unparseable GGUFs; pin NVFP4 gallery context_size by @localai-bot in https://github.com/mudler/LocalAI/pull/10563
- ci(vibevoice): skip the ASR transcription e2e on release tag builds by @localai-bot in https://github.com/mudler/LocalAI/pull/10567
- fix(gallery): match mmproj/model quant as a whole token so F16 no longer selects BF16 (#10559) by @localai-bot in https://github.com/mudler/LocalAI/pull/10564
- fix(distributed): return empty backend list for agent nodes instead of failing backend.list (#10545) by @localai-bot in https://github.com/mudler/LocalAI/pull/10565
- feat(distributed): add LOCALAI_DISTRIBUTED_SHARED_MODELS to skip staging on shared volumes (#10556) by @localai-bot in https://github.com/mudler/LocalAI/pull/10566
- chore: :arrow_up: Update leejet/stable-diffusion.cpp to
9956436c925a367daeab097598b1ea1f32d3503fby @localai-bot in https://github.com/mudler/LocalAI/pull/10533 - fix(openresponses): bound resume-stream buffer and enforce response ownership by @localai-bot in https://github.com/mudler/LocalAI/pull/10569
- chore: :arrow_up: Update ggml-org/whisper.cpp to
0ae02cdb2c7317b50991367c165736ce42ed96acby @localai-bot in https://github.com/mudler/LocalAI/pull/10532 - chore: :arrow_up: Update CrispStrobe/CrispASR to
6514c9da00b03a2f0f1b49a43fae4f3a01a41844by @localai-bot in https://github.com/mudler/LocalAI/pull/10535 - chore: :arrow_up: Update ggml-org/llama.cpp to
0ed235ea2c17a19fc8238668653946721ed136fdby @localai-bot in https://github.com/mudler/LocalAI/pull/10536 - fix(ik-llama): port multimodal path to mtmd API and bump to f96eaddb (#10534) by @localai-bot in https://github.com/mudler/LocalAI/pull/10568
- feat(backends): add voice-detect + face-detect ggml backends (replace Python insightface/speaker-recognition) by @localai-bot in https://github.com/mudler/LocalAI/pull/10441
- fix(kokoro): add explicit click dep so spacy CLI works on intel build by @localai-bot in https://github.com/mudler/LocalAI/pull/10572
- fix(launcher): robust binary download/upgrade (resume, rate-limit, UX) by @localai-bot in https://github.com/mudler/LocalAI/pull/10575
- fix(distributed): missing agent NATS permissions by @ALameLlama in https://github.com/mudler/LocalAI/pull/10571
- fix(fish-speech): allow invalid_reference_casting so tokenizers builds on darwin by @localai-bot in https://github.com/mudler/LocalAI/pull/10573
- fix(oci): retry layer downloads on transient network errors by @localai-bot in https://github.com/mudler/LocalAI/pull/10579
- chore(model-gallery): :arrow_up: update checksum by @localai-bot in https://github.com/mudler/LocalAI/pull/10585
- chore: :arrow_up: Update leejet/stable-diffusion.cpp to
c1790754d31bec0731ed5fddc9d5b9ff22ee19cdby @localai-bot in https://github.com/mudler/LocalAI/pull/10584 - chore: :arrow_up: Update CrispStrobe/CrispASR to
6b50f76e59700665358a1aabf5295597fa318e06by @localai-bot in https://github.com/mudler/LocalAI/pull/10583 - chore: :arrow_up: Update ggml-org/llama.cpp to
dbdaece23de9ac63f2e7ca9e6bfcdc4fc156a3faby @localai-bot in https://github.com/mudler/LocalAI/pull/10582 - chore: :arrow_up: Update mudler/voice-detect.cpp to
3d510772357538c5182808ac7de2278b84824e24by @localai-bot in https://github.com/mudler/LocalAI/pull/10581 - chore: :arrow_up: Update mudler/face-detect.cpp to
06914b077d52f90d5421299138e7be6bdd06b5e8by @localai-bot in https://github.com/mudler/LocalAI/pull/10580 - chore: :arrow_up: Update vllm-metal (darwin) to
v0.3.0.dev20260628073537by @localai-bot in https://github.com/mudler/LocalAI/pull/10562 - chore(recon): re-pin voice/face-detect to squashed release commits (+ graph-cache fix) by @localai-bot in https://github.com/mudler/LocalAI/pull/10591
- fix(sglang): parse tool_call function arguments before applying the chat template by @pos-ei-don in https://github.com/mudler/LocalAI/pull/10558
- feat(realtime): Semantic VAD EOU token by @richiejp in https://github.com/mudler/LocalAI/pull/10444
- fix(openai): stop max_tokens streaming retry loop on reasoning models (#9716) by @Dennisadira in https://github.com/mudler/LocalAI/pull/10448
- fix(import): derive model name from selected GGUF for repo-root URIs by @Dennisadira in https://github.com/mudler/LocalAI/pull/10589
- fix(functions): avoid quadratic-time debug logging in CleanupLLMResult / ParseFunctionCall by @pos-ei-don in https://github.com/mudler/LocalAI/pull/10592
- chore: :arrow_up: Update leejet/stable-diffusion.cpp to
3b6c9ca97cfcda8e68e719e6670d06379fcbe943by @localai-bot in https://github.com/mudler/LocalAI/pull/10594 - chore: :arrow_up: Update ggml-org/llama.cpp to
6f4f53f2b7da54fcdbbecaaa734337c337ad6176by @localai-bot in https://github.com/mudler/LocalAI/pull/10595 - chore: :arrow_up: Update localai-org/privacy-filter.cpp to
595f59630c69d361b5196f2aba2c71c873d0c13cby @localai-bot in https://github.com/mudler/LocalAI/pull/10596 - chore: :arrow_up: Update CrispStrobe/CrispASR to
3b93758f9725d400eca82976f895e4cec3f31260by @localai-bot in https://github.com/mudler/LocalAI/pull/10597 - chore: :arrow_up: Update ikawrakow/ik_llama.cpp to
f74a6fb87b315b2c3154166e075360e15021a61dby @localai-bot in https://github.com/mudler/LocalAI/pull/10598 - fix(import): strip file:// scheme from model path for local imports by @Dennisadira in https://github.com/mudler/LocalAI/pull/10599
- fix(tests): align openresponses test model name with GGUF-derived naming (#10589) by @localai-bot in https://github.com/mudler/LocalAI/pull/10609
- fix(macos): staple the notarization ticket to the .app, not just the dmg by @localai-bot in https://github.com/mudler/LocalAI/pull/10606
- fix(watchdog): persist UI-saved Check Interval across restarts (#10601) by @localai-bot in https://github.com/mudler/LocalAI/pull/10605
- feat(config): default swa_full:true for sliding-window-attention models by @localai-bot in https://github.com/mudler/LocalAI/pull/10611
New Contributors
- @ALameLlama made their first contribution in https://github.com/mudler/LocalAI/pull/10549
Full Changelog: https://github.com/mudler/LocalAI/compare/v4.5.5...v4.5.6