What local LLM are you using and which work best?

[mikevanes]

This discussion is a place to share your findings on which local LLMs work best with Norish and what the real-world experience is like when using them.

In particular, it would be great if the community could help identify:

• Models that perform well for Norish’s import and AI features
• The least resource-intensive models that still provide decent import speed and accuracy
• Practical tradeoffs between model size, speed, and hardware requirements

If you’re running Norish with a local LLM, please share:

• The model you’re using (name, size, quantization if relevant)

• The backend / inference engine

  • (e.g. Ollama, llama.cpp, LM Studio, vLLM, etc.)

• Hardware details

  • CPU / GPU, RAM, and where it’s running (local machine, server, NAS, etc.)

• Which Norish features you use with it

• How the experience has been

  • Speed, accuracy, stability

• Anything you’d recommend or avoid

The goal is to help the community converge on good baseline models and backends that work reliably with Norish without requiring excessive hardware.

[AfoxDesignz]

Not an LLM, but might still be helpful for some people: For the STT I use speaches.ai with Systran/faster-whisper-small.
It's pretty lightweight and runs smoothly on my ARM CPU. I haven't had any issues so far :)