v3.5.0 #6195

mudler · 2025-09-03T20:23:15Z

mudler
Sep 3, 2025
Maintainer

🚀 LocalAI 3.5.0

Welcome to LocalAI 3.5.0! This release focuses on expanding backend support, improving usability, refining the overall experience, and keeping reducing footprint of LocalAI, to make it a truly portable, privacy-focused AI stack. We’ve added several new backends, enhanced the WebUI with new features, made significant performance improvements under the hood, and simplified LocalAI management with a new Launcher app (Alpha) available for Linux and MacOS.

TL;DR – What’s New in LocalAI 3.5.0 🎉

🖼️ Expanded Backend Support: Welcome to MLX! mlx, mlx-audio, mlx-vlm are now all available in LocalAI. We also added support to WAN for video generation, and a CPU and MPS version of the diffusers backend! Now you can generate and edit images from MacOS or if you don't have any GPU (albeit slow).
✨ WebUI Enhancements: Download model configurations, a manual model refresh button, streamlined error streaming during SSE events, and a stop button for running backends. Models now can also be imported and edited via the WebUI.
🚀 Performance & Architecture: Whisper backend has been rewritten in Purego with integrated Voice Activity Detection (VAD) for improved efficiency and stability. Stablediffusion also benefits from the Purego conversion.
🛠️ Simplified Management: New LocalAI Launcher App (Alpha) for easy installation, startup, updates, and access to the WebUI.
✅ Bug Fixes & Stability: Resolutions to AMD RX 9060XT ROCm errors, libomp linking issues, model loading problems on macOS, CUDA device detection improvements, and more.
Enhanced support for MacOS: whisper, diffusers, llama.cpp, MLX (VLM, Audio, LLM), stable-diffusion.cpp will now work on MacOS!

What’s New in Detail

🚀 New Backends and Model Support

We've significantly expanded the range of models you can run with LocalAI!

mlx-audio: Bring text to life with Kokoro’s voice models on MacOS with the power of MLX!. Install with the mlx-audio backend. Example configuration:
```
backend: mlx-audio
name: kokoro-mlx
parameters:
  model: prince-canuma/Kokoro-82M
  voice: "af_heart"
  known_usecases:
    - tts
```

mlx-vlm: Experiment with the latest VLM models. While we don't have any models in the gallery, it's really easy to configure, see Add mlx-vlm #6119 for more details.

name: mlx-gemma
backend: mlx-vlm
parameters:
  model: "mlx-community/gemma-3n-E2B-it-4bit"
template:
  use_tokenizer_template: true
known_usecases:
- chat

WAN: Generate videos with Wan2.1 or Wan 2.2 models using the diffusers backend, supporting both I2V and T2V. Example configuration:

name: wan21
f16: true
backend: diffusers
known_usecases:
  - video
parameters:
  model: Wan-AI/Wan2.1-T2V-1.3B-Diffusers
diffusers:
  cuda: true
  pipeline_type: WanPipeline
  step: 40
options:
    - guidance_scale:5.0
    - num_frames:81
    - torch_dtype:bf16

Diffusers CPU and MacOS Support: Run diffusers models directly on your CPU without a GPU or with a Mac! This opens up LocalAI to a wider range of hardware configurations.

✨ WebUI Improvements

We've added several new features to make using LocalAI even easier:

Download Model Config: A "Get Config" button in the model gallery lets you download a model’s configuration file without installing the full model. This is perfect for custom setups and easier integration.
Manual Model Refresh: A new button allows you to manually refresh the on-disk YAML configuration, ensuring the WebUI always has the latest model information.
Streamlined Error Handling: Errors during SSE streaming events are now displayed directly to the user, providing better visibility and debugging information.
Backend Stop Button: Quickly stop running backends directly from the WebUI.

Model import and edit: Now models can be edited and imported directly from the WebUI.

Screenshot 2025-08-14 at 22-28-59 LocalAI - Import Model

Screenshot 2025-08-14 at 22-28-47 LocalAI - Edit Model gpt-oss-20b

Installed Backend List: Now displays installed backends in the WebUI for easier access and management.

🚀 Performance & Architecture Improvements

Purego Whisper Backend: The Whisper backend has been rewritten in Purego for increased performance and stability. This also includes integrated Voice Activity Detection (VAD) for detecting speech.
Purego Stablediffusion: Similar to Whisper, Stablediffusion has been converted to Purego, improving its overall architecture and enabling better compatibility.

🛠️ Simplified Management – Introducing the LocalAI Launcher (Alpha)

We're excited to introduce the first version of the LocalAI Launcher! This application simplifies:

Installation
Startup/Shutdown
Updates
Access to the WebUI and Application Folder

Please note: The launcher is in Alpha and may have bugs. The macOS build requires workarounds to run due to binaries not yet signed, and specific steps for running it are needed: https://discussions.apple.com/thread/253714860?answerId=257037956022#257037956022.

✅ Bug Fixes & Stability Improvements

AMD RX 9060XT ROCm Error: Fixed an issue causing errors with AMD RX 9060XT GPUs when using ROCm. This error, "ROCm error: invalid device function", occurred because of device function incompatibility. The fix involves updating the ROCm image and ensuring the correct GPU targets are specified during compilation. Recommended kernel versions and verification steps for GPU detection are available [here](link to troubleshooting doc if created).
libomp Linking: Resolved a missing libomp.so issue on macOS Docker containers.
macOS Model Loading: Addressed a problem where models could not be loaded on macOS. This was resolved by bundling necessary libutf8 libraries.
CUDA Device Detection: Improved detection of available GPU resources.
Flash Attention: Set auto for flash_attention in llama.cpp, allowing the system to optimize performance.

Additional Improvements

System Backend: Added a new "system" backend path (LOCALAI_BACKENDS_SYSTEM_PATH or via command-line arguments) defaulting to /usr/share/localai/backends. This allows specifying a read-only directory for backends, useful for package management and system-wide installations.
P2P Model Sync: Implemented automatic synchronization of installed models between LocalAI instances within a federation. Currently limited to models installed through the gallery, and configuration changes are not synced. Future improvements will address these limitations.
Diffusers Image Source Handling: Enhanced image source selection in the diffusers backend, prioritizing ref_images over src for more robust loading behavior.
Darwin CI Builds: Added support for building some Go-based backends (Stablediffusion and Whisper) on Darwin (macOS) in the CI pipeline.

🚨 Important Notes

Launcher (Alpha): The LocalAI Launcher is in its early stages of development. Please report any issues you encounter. The MacOS build requires additional steps due to code signing.
Model Configuration Updates: Changes to model configuration files are not currently synchronized when using P2P model sync.

The Complete Local Stack for Privacy-First AI

LocalAI	The free, Open Source OpenAI alternative. Acts as a drop-in replacement REST API compatible with OpenAI specifications for local AI inferencing. No GPU required. Link: https://github.com/mudler/LocalAI
LocalAGI	A powerful Local AI agent management platform. Serves as a drop-in replacement for OpenAI's Responses API, supercharged with advanced agentic capabilities and a no-code UI. Link: https://github.com/mudler/LocalAGI
LocalRecall	A RESTful API and knowledge base management system providing persistent memory and storage capabilities for AI agents. Designed to work alongside LocalAI and LocalAGI. Link: https://github.com/mudler/LocalRecall

Thank you! ❤️

A massive THANK YOU to our incredible community and our sponsors! LocalAI has over 35.000 stars, and LocalAGI has already rocketed past 1100+ stars!

As a reminder, LocalAI is real FOSS (Free and Open Source Software) and its sibling projects are community-driven and not backed by VCs or a company. We rely on contributors donating their spare time and our sponsors to provide us the hardware! If you love open-source, privacy-first AI, please consider starring the repository, contributing code, reporting bugs, or spreading the word!

Full changelog 👇

👉 Click to expand 👈

What's Changed

New Contributors

@lxnay made their first contribution in Tune the "dark gray" font color in the webui to make it more readable #6078
@mcowger made their first contribution in feat: Add a model refresh button to manually refresh on-disk yaml #6150

Full Changelog: v3.4.0...v3.5.0

This discussion was created from the release v3.5.0.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

v3.5.0 #6195

Uh oh!

{{title}}

Uh oh!

LocalAI

LocalAGI

LocalRecall

What's Changed

Bug fixes 🐛

Exciting New Features 🎉

🧠 Models

📖 Documentation and examples

👒 Dependencies

Other Changes

Replies: 0 comments

Select a reply

Uh oh!

Uh oh!

v3.5.0 #6195

Uh oh!

mudler Sep 3, 2025 Maintainer

🚀 LocalAI 3.5.0

TL;DR – What’s New in LocalAI 3.5.0 🎉

What’s New in Detail

🚀 New Backends and Model Support

✨ WebUI Improvements

🚀 Performance & Architecture Improvements

🛠️ Simplified Management – Introducing the LocalAI Launcher (Alpha)

✅ Bug Fixes & Stability Improvements

Additional Improvements

🚨 Important Notes

The Complete Local Stack for Privacy-First AI

LocalAI

LocalAGI

LocalRecall

Thank you! ❤️

Full changelog 👇

What's Changed

Bug fixes 🐛

Exciting New Features 🎉

🧠 Models

📖 Documentation and examples

👒 Dependencies

Other Changes

New Contributors

Replies: 0 comments

mudler
Sep 3, 2025
Maintainer