Skip to content

KnowledgeBase doesn't work with Llama-Sharp on local LLM #1086

@HHADI80

Description

@HHADI80

I have configured Llama-Sharp and I am using the llama3.1-8b-instruct-q6_k.gguf model on my cpu. I have installed the knowledgebase plugin and I have also installed and configured qdrant. The knowledge that I have installed in qdrant is of the Q&A type. When I ask a question in the chat, Llama Sharp does not use knowledge and gives the user the answer through LLM. I have put the agent configuration below.
{
"id": "121a6dab-afa9-469f-bbe6-8df111b77700",
"name": "Q&A bot",
"description": "This is a Q&A bot",
"type": "task",
"mode": "eager",
"createdDateTime": "2025-05-10T12:45:57.3500389Z",
"updatedDateTime": "2025-07-02T09:52:29.17704Z",
"llmConfig": {
"is_inherit": false,
"provider": "llama-sharp",
"model": "llama3.1-8b-instruct-q6_k.gguf",
"max_recursion_depth": 16,
"max_output_tokens": 4096
},
"isPublic": true,
"disabled": false,
"iconUrl": null,
"profiles": [],
"labels": [],
"mergeUtility": false,
"utilities": [
{
"name": "kg.knowledge-base",
"disabled": false,
"functions": [
{
"name": "util-kg-knowledge_retrieval"
}
],
"templates": [
{
"name": "util-kg-knowledge_retrieval.fn"
}
]
}
],
"mcpTools": [],
"rules": [],
"knowledgeBases": [
{
"name": "BotSharp",
"type": "question-answer",
"disabled": false,
"confidence": 0.1
}
],
"inheritAgentId": null,
"maxMessageCount": 4096,
"routingRules": []
}

And the Instruct was as follows:
"
You are a helpful Q&A assistant. When the user asks a question, search the knowledge base using the 'util-kg-knowledge_retrieval' function to retrieve relevant information, then respond clearly and helpfully based on the retrieved knowledge. If no relevant knowledge is found, politely inform the user. Always keep answers accurate and easy to understand.

Use this function for retrieving knowledge "util-kg-knowledge_retrieval"
"

I ask the experts to help me solve this problem.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions