Issue recognizing tool for local LLM

[RichardHooijmaijers]

I am running into an issue when following the example on the pkgdown.
The only thing that was adapted is usage of ollama instead of chatgtp. All went well up to the point where the actual chat call is given:

library(ragnar)
#> Warning: package 'ragnar' was built under R version 4.5.1
store_location <- "r4ds.ragnar.duckdb"
store <- ragnar_store_connect(store_location, read_only = TRUE)
chat <- ellmer::chat_ollama(base_url = "http://localhost:11434",  model="llama3.2")
ragnar_register_tool_retrieve(chat, store, top_k = 10)
chat$chat("How can I subset a dataframe?")
#> ◯ [tool call] pd.DataFrame.loc(df = "True", labels = "None", loc = "[0:5,
#> 0:3]")
#> ■ #> Error: Unknown tool

The store was created as

store <- ragnar_store_create(
  store_location,
  embed = \(x) ragnar::embed_ollama(x, model = "snowflake-arctic-embed2:568m", base_url = "http://localhost:11434")
)

I get an answer but without using the store. Any help is greatly appreciated!

[t-kalinowski]

The Llama models served via ollama are not very good at tool calling in my experience. Anactdotally, I've had better luck with qwen3, but I've not done exhaustive testing and the landscape changes regularly. If you're using ollama, I would consult this listing, filtered on "tools" and try some of the other models here: https://ollama.com/search?c=tools&o=newest.

We have some ideas about how to better support models that don't support tool calling, or other ways to insert store retrieved results besides via tools. Some WIP branches in #55 and t-kalinowski/quartohelp#8.

If you end up writing something custom and running some experiments, I'd be interested to hear what worked well for you and what didn't!

[t-kalinowski]

The gpt-oss models are the best open models at tool calling today I believe.