Goose with Docker Model Runner (DMR)

[scaprile]

Hi,
I'm an embedded systems developer, but totally null with AI.
Have DMR running fine, can chat with it on its CLI, I'm using a gemma model.
Configured Goose to connect to the docker server.
After a couple of times of getting nowhere, I started Wireshark on port 12434 in order to see what's going on.
I just write "Hello", and send. (In Goose chat)
Goose sends a >20K query to the server, which responds with a JSON telling context has been exceeded.
Goose shows me that and says it is compacting the chat
Goose sends a >3K query requesting a model "gpt-4o-mini" that of course I don't have, how will I know ?
Goose tells me it can't go on and that's the end of my front-end experience.
gpt-4o-mini is not in the config file, I can't change it.
Docker doesn't seem to have that model in ai/
This is capture file "take1"
Can't find a model with that exact name in Huggingface, pull one that says it is that, iterate until one of those has a reasonable size that justifies the word "mini", most have the same size as the main one I intend to run...
Tag the model as the required name, test it on Docker, fine on CLI
Try again, same result, Docker reports context exceeded again
This is capture file "take2"
End of my experience.
I don't have a clue where to set the context. Yeah, quite likely it is Docker's, not Goose's. If you can help me: fine; just take care of that chicken and egg issue with models and try to be more verbose so a non tech-savvy user can get what is going on. (Yeah, non-tech-savvy users won't install their own LLM... you're right...)
Wireshark_capture_files.zip
diagnostics_20260201_1.zip

[scaprile]

$ docker model configure --context-size 8192 <model>

Does the trick.

Now I'm greeted by an internal error...

[diagnostics_20260201_1.zip](https://github.com/user-attachments/files/24995022/diagnostics_20260201_1.zip)

[michaelneale]

hi @scaprile quite a few variables - but that error seems that somehow the protocol has broken down (likely a model that is swamped by context). A context window of > 32k is realistically needed to even start, 8K won't really do it and may result in things being truncated which possibly could cause this? you are using the docker model runner provider perhaps?

can oyu provide the exact GOOSE_PROVIDER and _MODEL from ~/.config/goose/config.yaml?

[scaprile]

Hi @michaelneale , thx 4 your time.
Well, in my searches I bumped into suggestions for the model and instructions to set the provider to OpenAI. This is my setup:

GOOSE_PROVIDER: openai
GOOSE_MODEL: hf.co/unsloth/gemma-3n-e4b-it-gguf:q6_k
GOOSE_MODE: approve
OPENAI_HOST: http://localhost:12434
OPENAI_BASE_PATH: /engines/llama.cpp/v1/chat/completions
OPENAI_API_KEY: irrelevant
OPENAI_TIMEOUT: '600'

The model is just what was suggested in some tutorial I can't remember, I'm open to use any model that works... yesterday I tried to get something useful from this gemma from the CLI, and it can only hallucinate and apologize... and talk and talk... oh my...

I raised the context to 65536 and now the error response is immediate.
Wireshark: Goose request says role: system, so that error response is rejecting that role...

I'm quite open to use ANY model you suggest, I just need an assistant to search for stuff, maybe review some code (maybe, not actually needed), and help with repetitive (and boring) tasks.

[scaprile]

you are using the docker model runner provider perhaps?

As that is in the title of the question, I guess you are asking something else... though I don't know what.

[michaelneale]

ah sorry, I thought there was a DMR provider in goose specifically, but openai compatible seems ok. Hrm let me have a think and play.

[michaelneale]

I think if you do something like:

docker model configure --context-size 65535 ...

then it can work with an agent like goose - the default seems rather small at 8k.

[scaprile]

I raised the context to 65536

That is exactly what I did.
Then, in Wireshark, I could see that

Goose request says role: system, so that error response is rejecting that role...

In other words:

Goose: "role": "system"
Model: "error": "Conversation roles must alternate user/assistant/...

In other words: they don't understand each other.

[scaprile]

I raised the context to 65536

That is exactly what I did.
Then, in Wireshark, I could see that

Goose request says role: system, so that error response is rejecting that role...

In other words:

Goose: "role": "system"
Model: "error": "Conversation roles must alternate user/assistant/...

In other words: they don't understand each other.

[scaprile]

Solved by pulling ai/qwen3:30B-A3B-Q4_K_M instead, and also setting its context to 64K