Comparison of small open source LLMs (8b parameters or less)
General | Tool Usage | Chat Room | Logic | Vision | Vision OCR | Code Generation | More
All testing done with Ollama
Key | |
---|---|
✅ | Pass |
☑️ | Partial Pass |
❌ | Fail |
🕒 | Timeout |
😵💫 | Hallucinations |
🚫 | Rejected prompt / unsafe content |
💔 | Malformed Response / Tool call |
Tested with attogram/ollama-multirun
Small Model | Hovercraft | Pirate | Who |
---|---|---|---|
bakllava:7b | ❌ | ✅ | |
codellama:7b | ✅ | ✅ | ✅ |
cogito:3b | ✅ | ✅ | |
cogito:bb | ✅ | ✅ | |
deepcoder:1.5b | ✅ | ☑️ | ✅ |
deepseek-r1:1.5b | ✅ | ☑️ | ✅ |
deepseek-r1:8b | ☑️ | ✅ | ✅ |
dolphin-mistral:7b | ☑️ | ✅ | ✅ |
dolphin3:8b | ✅ | ✅ | ✅ |
gemma3:1b | ☑️ | ✅ | ✅ |
gemma3:4b | ☑️ | ✅ | |
gemma:2b | ☑️ | ✅ | |
granite3.2-vision:2b | ✅ | ✅ | |
granite3.3:2b | ☑️ | ✅ | |
huihui_ai/baronllm-abliterated:8b | ☑️ | ✅ | |
llama3-groq-tool-use:8b | ☑️ | ✅ | |
llama3.2:1b | ✅ | ☑️ | |
llava-llama3:8b | ☑️ | ✅ | |
llava-phi3:3.8b | ✅ | ✅ | |
llava:7b | ✅ | ✅ | |
minicpm-v:8b | ❌ | ✅ | |
mistral:7b | ☑️ | ✅ | |
qwen2.5-coder:7b | ✅ | ✅ | |
qwen2.5vl:3b | ✅ | ✅ | |
qwen2.5vl:7b | ☑️ | ✅ | |
qwen3:1.7b | ✅ | 🕒 | ✅ |
qwen3:4b | ✅ | 🕒 | ✅ |
qwen3:8b | ☑️ | ✅ | ✅ |
smollm2:135m | ✅ | ✅ | |
smollm2:360m | ☑️ | ✅ | |
smollm2:1.7b | ✅ | ✅ | |
stable-code:3b | ☑️ | ☑️ | ✅ |
starcoder:7b | ❌ | ❌ | ❌ |
Small Model | Hovercraft | Pirate | Who |
deepseek-r1:7b
llama3.1:8b
llama3.3:3b
llama3:8b
phi:2.7b
phi3:3.8b
tinyllama:1.1b
qwen3:0.6b
qwen3:4b
Tested with attogram/ollama-bash-toolshed
Passing | Math | Ollama | Time | Web |
---|---|---|---|---|
cogito:3b | ✅ | ✅ | ✅ | ✅ |
cogito:8b | ✅ | ✅ | ✅ | ✅ |
granite3.3:2b | ✅ | ✅ | ✅ | ✅ |
hermes3:8b | ✅ | ✅ | ✅ | ✅ |
llama3.2:1b | ✅ | ✅ | ✅ | ✅ |
llama3.2:3b | ✅ | ✅ | ✅ | ✅ |
qwen3:1.7b | ✅ | ✅ | ✅ | ✅ |
qwen3:4b | ✅ | ✅ | ✅ | ✅ |
qwen3:8b | ✅ | ✅ | ✅ | ✅ |
Failing | Math | Ollama | Time | Web |
---|---|---|---|---|
command-r7b:7b | ❌ | ❌ | ❌ | ❌ |
granite3.2-vision:2b | ❌ | ❌ | ❌ | ❌ |
granite3.3:8b | 💔 | 💔 | 💔 | 💔 |
llama3-groq-tool-use:8b | ✅ | ✅ | ❌ | ✅ |
mistral:7b | 💔 | 💔 | 💔 | 💔 |
nemotron-mini:4b | ✅ | ❌ | ✅ | ☑️ |
qwen2.5-coder:7b | 💔 | 💔 | 💔 | 💔 |
qwen3:0.6b | ✅ | ☑️ | ❌ | ✅ |
smollm2:1.7b | 💔 | 💔 | ✅ | 😵💫 |
aya-expanse:8b
command-r7b-arabic:7b
granite3-dense:2b
granite3-dense:8b
granite3-moe:1b
granite3-moe:3b
granite3.1-dense:2b
granite3.1-dense:8b
granite3.1-moe:1b
granite3.1-moe:3b
granite3.2:2b
granite3.2:8b
hermes3:3b
llama3.1:8b
mixtral:8x7b
phi4-mini:3.8b
qwen2.5-coder:0.5b
qwen2.5-coder:1.5b
qwen2.5-coder:3b
qwen2.5:0.5b
qwen2.5:1.5b
qwen2.5:3b
qwen2.5:7b
qwen2:0.5b
qwen2:1.5b
qwen2:7b
Hituzip/gemma3-tools:4b
PetrosStav/gemma3-tools:4b
hhao/qwen2.5-coder-tools:0.5b
hhao/qwen2.5-coder-tools:1.5b
hhao/qwen2.5-coder-tools:3b
hhao/qwen2.5-coder-tools:7b
ishumilin/deepseek-r1-coder-tools-tuned:1.5b
ishumilin/deepseek-r1-coder-tools-tuned:7b
ishumilin/deepseek-r1-coder-tools-tuned:8b
ishumilin/deepseek-r1-coder-tools:1.5b
ishumilin/deepseek-r1-coder-tools:7b
ishumilin/deepseek-r1-coder-tools:8b
maryasov/qwen2.5-coder-cline:7b
orieg/gemma3-tools:1b
orieg/gemma3-tools:4b
tom_himanen/deepseek-r1-roo-cline-tools:1.5b
tom_himanen/deepseek-r1-roo-cline-tools:7b
tom_himanen/deepseek-r1-roo-cline-tools:8b
Tested with attogram/llm-council
Small Model | Peace | Quit | Topic |
---|---|---|---|
bakllava:7b | ❌ | ||
codellama:7b | ✅ | ||
deepcoder:1.5b | ❌ | ||
deepseek-r1:1.5b | ✅ | ❌ | |
deepseek-r1:8b | ✅ | ||
dolphin-mistral:7b | ✅ | ||
dolphin3:8b | ✅ | ✅ | |
gemma3:1b | ✅ | ||
gemma3:4b | ✅ | ✅ | ✅ |
gemma:2b | ✅ | ||
granite3.2-vision:2b | ❌ | ||
granite3.3:2b | ✅ | ✅ | ❌ |
huihui_ai/baronllm-abliterated:8b | ✅ | ||
llama3-groq-tool-use:8b | ✅ | ||
llama3.2:1b | ❌ | ||
llava-llama3:8b | ✅ | ||
llava-phi3:3.8b | ✅ | ||
llava:7b | ✅ | ||
minicpm-v:8b | ✅ | ✅ | |
mistral:7b | ✅ | ✅ | |
qwen2.5-coder:7b | ✅ | ||
qwen2.5vl:3b | ✅ | ||
qwen2.5vl:7b | ✅ | ||
qwen3:1.7b | ✅ | ||
qwen3:8b | ✅ | ✅ | |
smollm2:135m | ❌ | ||
smollm2:360m | ❌ | ||
smollm2:1.7b | ❌ | ||
stable-code:3b | ❌ | ||
starcoder:7b | ❌ | ||
Small Model | Peace | Quit | Topic |
Tested with attogram/ollama-multirun
Small Model | Socrates | Strawberry |
---|---|---|
bakllava:7b | ✅ | ❌ |
❌ | ❌ | |
deepcoder:1.5b | ✅ | ✅ |
deepseek-r1:1.5b | ✅ | ✅ |
deepseek-r1:8b | ✅ | 🕒 |
dolphin-mistral:7b | ✅ | ❌ |
dolphin3:8b | ✅ | ❌ |
gemma3:1b | ✅ | ✅ |
gemma3:4b | ✅ | ✅ |
gemma:2b | ✅ | ✅ |
granite3.2-vision:2b | ✅ | ✅ |
granite3.3:2b | ✅ | ✅ |
huihui_ai/baronllm-abliterated:8b | ✅ | ❌ |
llama3-groq-tool-use:8b | ✅ | ✅ |
❌ | ❌ | |
llava-llama3:8b | ✅ | ❌ |
llava-phi3:3.8b | ✅ | ❌ |
llava:7b | ✅ | ❌ |
minicpm-v:8b | ✅ | ❌ |
mistral:7b | ✅ | ❌ |
qwen2.5-coder:7b | ✅ | ❌ |
qwen2.5vl:3b | ✅ | ✅ |
qwen2.5vl:7b | ✅ | ❌ |
qwen3:1.7b | ✅ | ✅ |
qwen3:8b | ✅ | ✅ |
smollm2:135m | ☑️ | ❌ |
smollm2:360m | ✅ | ❌ |
smollm2:1.7b | ✅ | ❌ |
❌ | ❌ | |
❌ | ❌ | |
Small Model | Socrates | Strawberry |
Tested with attogram/ollama-multirun
Passing | Describe | Keywords | Faces |
---|---|---|---|
bakllava:7b | ✅ | ❌ | ❌ |
granite3.2-vision:2b | ✅ | ✅ | ❌ |
minicpm-v:8b | ✅ | ✅ | ☑️ |
qwen2.5vl:3b | ✅ | ✅ | ❌ |
qwen2.5vl:7b | ✅ | ✅ | ✅ |
gemma3:4b | ✅ | ✅ | ✅ |
llava:7b | ✅ | ✅ | ✅ |
llava-llama3:8b | ✅ | ☑️ | ☑️ |
llava-phi3:3.8b | ✅ | ☑️ | ☑️ |
moondream:1.8b | ✅ | ❌ | ❌ |
Tested with attogram/ollama-multirun
Passing | OCR |
---|---|
granite3.2-vision:2b | ☑️ |
minicpm-v:8b | ☑️ |
qwen2.5vl:3b | ✅ |
qwen2.5vl:7b | ✅ |
Failing | OCR |
---|---|
❌ | |
gemma3:4b | 🕒 |
❌ | |
❌ | |
❌ | |
❌ |
Small Model | bash-1 | js-1 | php-1 | python-1 |
---|---|---|---|---|
bakllava:7b | ❌ | ☑️ | ||
codellama:7b | ✅ ️ | ✅ | ||
deepcoder:1.5b | ❌ | ✅ | ||
deepseek-r1:1.5b | ❌ | ✅ | ||
deepseek-r1:8b | 🕒 | 🕒 | ||
dolphin-mistral:7b | ✅ | ✅ | ||
dolphin3:8b | ✅ | ✅ | ||
gemma3:1b | ☑️ | ☑️ | ||
gemma3:4b | ☑️ | ✅ | ||
gemma:2b | ✅ | ❌ | ||
granite3.2-vision:2b | ✅ | ✅ | ||
granite3.3:2b | ☑️ | ✅ | ||
huihui_ai/baronllm-abliterated:8b | ✅ | ✅ | ||
llama3-groq-tool-use:8b | ✅ | ✅ | ||
llama3.2:1b | ☑️ | ☑️ | ||
llava-llama3:8b | ☑️ | ✅ | ||
llava-phi3:3.8b | ✅ | ✅ | ||
llava:7b | ✅ | ✅ | ||
minicpm-v:8b | ❌ | ✅ | ||
mistral:7b | ☑️ | ✅ | ||
qwen2.5-coder:7b | ✅ | ✅ | ||
qwen2.5vl:3b | ☑️ | ✅ | ||
qwen2.5vl:7b | ✅ | ✅ | ||
qwen3:1.7b | ✅ | ✅ | ||
qwen3:8b | ✅ | ✅ | ||
❌ | ❌ | |||
smollm2:360m | ☑️ | ☑️ | ||
smollm2:1.7b | ✅ | ☑️ | ||
stable-code:3b | ☑️ | ✅ | ||
starcoder:7b | ☑️ | ❌ | ||
Small Model | bash-1 | js-1 | php-1 | python-1 |
Project | About |
---|---|
Attogram Project Discord Channel |
Join the Attogram Project Discord Channel for: - Announcements - Technical Support - General Chat about Attogram Projects |
Ollama Multirun | Run a prompt against all, or some, of your models running on Ollama. - Creates web pages with the output, performance statistics and model info. - All in a single Bash shell script. |
Ollama Bash Lib | A Bash Library to interact with Ollama |
Ollama Bash Toolshed | Chat with tool calling models. - Sample tools included. - Add new tools to your shed with ease. - Runs on Ollama. - All via Bash shell scripts. |
LLM Council | Start a chat room between all, or some, of your models running on Ollama. - All in a single Bash shell script. |
Small Models | Comparison of small open source LLMs - 8b parameters or less |
AI Test Zone | AI Testing reports - hosted on https://attogram.github.io/ai_test_zone/ |