Ollama (LLM)

The Large Language Model (LLM) component uses Ollama’s framework to provide access to a wide selection of available language models. It runs in its own Docker container, which is built during setup using the entrypoint.sh script.

For Docker, the Ollama base image is used to build the container, and the model types are defined inside the entrypoint.sh script. There is no dedicated Python script required to run the LLM itself, as most of the setup and model loading is handled by Docker during the build process.

Prompting must be predefined before the model is executed. For this reason, both prompt construction and model calls are handled by the Manager component. For more details on prompting logic, refer to the Manager wiki page.

Calling the Model

As mentioned previously, the LLM is called from the Manager component. The calls are made in app.py (lines 145 and 195), depending on which workflow is being executed. Based on the workflow, the appropriate model and prompt are selected and sent through an HTTP request to the Ollama API endpoint: OLLAMA_URL = 'http://ollama:11434/api/generate'

    try:
        ollama_response = requests.post(
            OLLAMA_URL,
            json={"prompt": prompt_text, "model": "gemma3:27b", "stream": False, "think": False}
        )
        ollama_response.raise_for_status()
        response_text = ollama_response.json().get("response", "")
        response_text = strip_avatar_prefix(response_text)
        print(response_text)

Changing Models

To change the models used by Ollama, two files need to be updated. First, navigate to the Ollama folder and open entrypoint.sh. On line 3 or 4, update the model variables to the desired models:

CONVERSATION_MODEL="gemma3:27b"
FEEDBACK_MODEL="qwen3:32b"

These two models serve different purposes:

• Conversation model is used during the conversation workflow
• Feedback model is used during the feedback workflow

If you change the conversation model, you must also update the model parameter in the Manager component. Open app.py in the Manager folder and modify the model value around line 163:

ollama_response = requests.post(
  OLLAMA_URL,
  json={"prompt": prompt_text, "model": "gemma3:27b", "stream": False, "think": False}
)

If you change the feedback model, update the corresponding call in the same file around line 214:

ollama_response = requests.post(
  OLLAMA_URL,
  json={"prompt": feedback_prompt, "model": "qwen3:32b", "stream": False, "think": False}
)

For a full list of available models, refer to the Ollama model library