[Question]: LLM on Ollama endpoints is not responding based on a document imported from the RAG API.

[SEOLJINYOUNG]

What is your question?

LLM on Ollama endpoints is not responding based on a document imported from the RAG API.

EMBEDDINGS_PROVIDER=ollama
EMBEDDINGS_MODEL=nomic-embed-text
MODEL=llama3.1:70b

I took a log app\clients\OllamaClient.js,
and on the messages list
It contains two messages.
messages = [
{role: 'system', content:'content from rag api'},
{role: 'user', content:'user input'},
]

More Details

When I ask you to tell me the contents of the attached document, llm's answer says that you don't save or remember the previous conversation, or answers based on existing pre-learned content.

I'd appreciate it if you could give me some ideas on which parts to look at.

What is the main subject of your question?

Endpoints, User System/OAuth, Other

Screenshots

Code of Conduct

• I agree to follow this project's Code of Conduct

[lokdao]

Hi, I am facing a similar issue.
I am serving llama3.2 via Ollama and am also using the nomic-embed-text through Ollama.
When uploading a document, llama3.2 does not detect it. However using ChatGPT-4o via Azure Open AI with the same file works perfectly fine.

Config

docker-compose-override.yaml

.env

Prompt

[aleibovici]

I have the exact same issue with a Ollama and uploaded documents.
If I change from Ollama to OpenAI, I receive an adequate response.
That tells me that the RAG API with nomic-embedding is working.

It will only work if I force-prompt, but that's not a solution.

[parthpat12]

I’m having same issue. Is there any workaround or fix?

[SEOLJINYOUNG]

@parthpat12
I changed the model and modified the prompts that configure the document content.
llama3->gemma2

Try changing the model first without modifying the prompt.

[parthpat12]

Changing model did work. Thanks

[litznerski]

Same problem here :-( It is impossible to get rid of openAPI when using RAG_API - impossible to use Ollama for Embeddings. A Perplexity Deep Research shows, that this is a common issue - seems to be hardcoded somewhere.