How to choose a model?

[outbound]

I followed the instructions and set up Minima locally to use with Claude desktop (MCP).
My goal is to use my docs as a RAG

Two questions

1. If I have two of 4 subdirectories in a folder that I want to use, do I specify both, or just the parent folder

2. Model Selection

I'm not sure about the model selections I made.

I chose the Ollama gemma3:27b model -
And here are my questions
2-1. Do I need to download this model?
2-2. Given I have an RTX3060 with 12GB RAM - do I download the K4 quant version, and do I specify that somewhere in minima's config.
2-3. Embedding size - what's the difference between

Here are my current selection:
Model: gemma3:27b
Size: 768
Embedding Model ID: sentence-transformers/all-mpnet-base-v2
Reranker: BAAI/bge-reranker-base

Essentially, I'm not sure on what to do. Can you please help?

[dmayboroda]

Hey @outbound
Thank you for questions

1. You need to specify only parent folder, all subfolders will be crawled automatically

2.1 You do not need to do it by your own, when you setup a model in .env file Ollama container automatically download it from Ollama registry. All that you need to check, does Ollama repository contains a model with such name.
2.2 No, all that yo need to do - just write a name of this model in .env, but again, pleasde be sure that Ollama repository contains suchmodel.
2.3 You have to variables to setup (embedding size and embedding model), The embedding size is the size of the output that the embedding model provide, for example sentence-transformers/all-mpnet-base-v2 model provide an embedding with a length of 768, so you specify Embedding Model ID: sentence-transformers/all-mpnet-base-v2 and embedding size: 768

So, your .env file must look like this
LOCAL_FILES_PATH=path to your parent folder
EMBEDDING_MODEL_ID=sentence-transformers/all-mpnet-base-v2
EMBEDDING_SIZE=768
OLLAMA_MODEL=gemma3:27b
RERANKER_MODEL=BAAI/bge-reranker-base