Having Problem registering anthropic Models on Llama-Stack

[mansiUnifynd]

I’ve been trying to register Anthropic models, but every time I add one it says the model isn’t supported.
Could someone please guide me on how to properly register Anthropic models on Llama Stack? It would be a great help.
Heres the list of things I have done:

1. I have update the API Key in my run.yaml
   - provider_id: anthropic
   provider_type: remote::anthropic
   config:
   api_key: ${env.ANTHROPIC_API_KEY:""}

2. Below is the command that i am using to register the model:
   llama-stack-client models register claude-3-5-sonnet --provider-id anthropic --provider-model-id claude-3-5-sonnet-latest --metadata {} --model-type llm

ERROR I AM GETTING:
INFO:httpx:HTTP Request: POST http://localhost:8321/v1/models "HTTP/1.1 400 Bad Request"
╭─────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Failed to register model │
│ │
│ Error Type: BadRequestError │
│ Details: Error code: 400 - {'detail': 'Invalid value: Unsupported model: claude-3-5-sonnet-latest'} │

[mattf]

check which models Anthropic's model listing endpoint (https://api.anthropic.com/v1/models) is advertising.

$ uv run --with llama-stack-client llama-stack-client models list

Available Models

┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━┓
┃ model_ty… ┃ identifier                           ┃ provider_resource_id       ┃ metadata ┃ provider… ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━┩
│ llm       │ anthropic/claude-sonnet-4-5-20250929 │ claude-sonnet-4-5-20250929 │          │ anthropic │
├───────────┼──────────────────────────────────────┼────────────────────────────┼──────────┼───────────┤
│ llm       │ anthropic/claude-opus-4-1-20250805   │ claude-opus-4-1-20250805   │          │ anthropic │
├───────────┼──────────────────────────────────────┼────────────────────────────┼──────────┼───────────┤
│ llm       │ anthropic/claude-opus-4-20250514     │ claude-opus-4-20250514     │          │ anthropic │
├───────────┼──────────────────────────────────────┼────────────────────────────┼──────────┼───────────┤
│ llm       │ anthropic/claude-sonnet-4-20250514   │ claude-sonnet-4-20250514   │          │ anthropic │
├───────────┼──────────────────────────────────────┼────────────────────────────┼──────────┼───────────┤
│ llm       │ anthropic/claude-3-7-sonnet-20250219 │ claude-3-7-sonnet-20250219 │          │ anthropic │
├───────────┼──────────────────────────────────────┼────────────────────────────┼──────────┼───────────┤
│ llm       │ anthropic/claude-3-haiku-20240307    │ claude-3-haiku-20240307    │          │ anthropic │
└───────────┴──────────────────────────────────────┴────────────────────────────┴──────────┴───────────┘

you can use anything in the provider_resource_id column.

[mansiUnifynd]

Hey Yes I did that. I tried registering all the models listed for my anthropic api endpoint. But I keep getting the same error every time.
I am able to register openai, gemini as well. I am only facing issue with anthropic.

Traceback (most recent call last):
File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.10/site-packages/llama_stack/distribution/server/server.py", line 597, in
main()
File "/usr/local/lib/python3.10/site-packages/llama_stack/distribution/server/server.py", line 487, in main
impls = asyncio.run(construct_stack(config))
File "/usr/local/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/local/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
return future.result()
File "/usr/local/lib/python3.10/site-packages/llama_stack/distribution/stack.py", line 231, in construct_stack
await register_resources(run_config, impls)
File "/usr/local/lib/python3.10/site-packages/llama_stack/distribution/stack.py", line 104, in register_resources
await method(**{k: getattr(obj, k) for k in obj.model_dump().keys()})
File "/usr/local/lib/python3.10/site-packages/llama_stack/providers/utils/telemetry/trace_protocol.py", line 103, in async_wrapper
result = await method(self, *args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/llama_stack/distribution/routing_tables/models.py", line 75, in register_model
registered_model = await self.register_object(model)
File "/usr/local/lib/python3.10/site-packages/llama_stack/distribution/routing_tables/common.py", line 196, in register_object
registered_obj = await register_object_with_provider(obj, p)
File "/usr/local/lib/python3.10/site-packages/llama_stack/distribution/routing_tables/common.py", line 37, in register_object_with_provider
return await p.register_model(obj)
File "/usr/local/lib/python3.10/site-packages/llama_stack/providers/utils/telemetry/trace_protocol.py", line 103, in async_wrapper
result = await method(self, *args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/llama_stack/providers/utils/inference/litellm_openai_mixin.py", line 94, in register_model
raise ValueError(f"Unsupported model: {model.provider_resource_id}")
ValueError: Unsupported model: claude-3-haiku-20240307

[mattf]

what's the stack trace with py3.12 and llama stack 0.3.0?

[mansiUnifynd]

To avoid the error I am getting below I reinstalled the older version. This is just a temporary Fix. But what i am not able to figure out is why is only not working for anthropic. Its working perfectly fine for openai, llama.cpp, openrouter. The only provider that is giving error is anthropic.

python3 -m pip show llama-stack-client

Name: llama_stack_client
Version: 0.2.9
Summary: The official Python library for the llama-stack-client API
Home-page: https://github.com/meta-llama/llama-stack-client-python
Author:
Author-email: Meta Llama [email protected]
License-Expression: Apache-2.0
Location: /Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages
Requires: anyio, click, distro, httpx, pandas, prompt-toolkit, pyaml, pydantic, rich, sniffio, termcolor, tqdm, typing-extensions
Required-by: llama_stack

-m pip show llama-stack
Name: llama_stack
Version: 0.2.9
Summary: Llama Stack
Home-page: https://github.com/meta-llama/llama-stack
Author:
Author-email: Meta Llama [email protected]
License: MIT
Location: /Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages
Requires: aiohttp, fire, h11, httpx, huggingface-hub, jinja2, jsonschema, llama-stack-client, openai, pillow, prompt-toolkit, pydantic, python-dotenv, python-jose, requests, rich, setuptools, starlette, termcolor, tiktoken
Required-by: llama-stack-haystack, llama-stack-provider-llamacpp, ramalama-stack

python3 --version
Python 3.13.3

[mansiUnifynd]

This is how my current yaml looks like. Its working perfectly fine for openai but not for anthropic.

providers:
inference:

- provider_id: openai
  provider_type: remote::openai
  config:
    api_key: "" 

- provider_id: anthropic
  provider_type: remote::anthropic
  config:
    api_key: "" 

models:

• model_id: gpt-4o-mini
  provider_id: openai
  provider_model_id: gpt-4o-mini

• model_id: anthropic/claude-opus-4-1-20250805. # Also tried adding just "claude-opus-4-1-20250805" here.
  provider_id: anthropic
  provider_model_id: claude-opus-4-1-20250805

Error:
raise ValueError(f"Unsupported model: {model.provider_resource_id}")
ValueError: Unsupported model: claude-opus-4-1-20250805

[mattf]

you should confirm which models your key has access to -

$ curl -H "x-api-key: $ANTHROPIC_API_KEY" -H "anthropic-version: 2023-06-01" https://api.anthropic.com/v1/models | jq -r '.data[].id'
claude-haiku-4-5-20251001
claude-sonnet-4-5-20250929
claude-opus-4-1-20250805
claude-opus-4-20250514
claude-sonnet-4-20250514
claude-3-7-sonnet-20250219
claude-3-5-haiku-20241022
claude-3-haiku-20240307

[mansiUnifynd]

curl -H "x-api-key: $ANTHROPIC_API_KEY" -H "anthropic-version: 2023-06-01" https://api.anthropic.com/v1/models | jq -r '.data[].id'

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--100 1052 100 1052 0 0 2778 0 --:--:-- --:--:-- --:--:-- 2783
claude-haiku-4-5-20251001
claude-sonnet-4-5-20250929
claude-opus-4-1-20250805
claude-opus-4-20250514
claude-sonnet-4-20250514
claude-3-7-sonnet-20250219
claude-3-5-haiku-20241022
claude-3-haiku-20240307

[mansiUnifynd]

Also I am facing another issue where I want to use a filesystem mcp server hosted on port 8000. But everytimne I run the code I get an error that my sever is not compatible with my client.
The Distribution I am using on docker is llamastack/distribution-ollama:latest

Code:

import json
import uuid
from llama_stack_client import LlamaStackClient, Agent
from llama_stack_client.types import UserMessage

client = LlamaStackClient(base_url="http://localhost:8321")

llm = next(
m
for m in client.models.list()
if m.model_type == "llm" and m.provider_id == ""
)
model_name = llm.identifier

client.toolgroups.register(
toolgroup_id="mcp::filesystem",
provider_id="model-context-protocol-1",
mcp_endpoint="http://localhost:8000/sse",
)

agent = Agent(
agent_id=f"agent-{uuid.uuid4().hex[:8]}",
model=model_name,
instructions=(
"You are a helpful assistant that can read/write files via the MCP "
"filesystem tool."
),
tools=["mcp::filesystem"],
extra_headers={
"X-LlamaStack-Provider-Data": json.dumps(
{
"mcp_headers": {
"http://localhost:8000/sse": {}
}
}
)
},
)

print("Agent created:", agent.agent_id)

turn = agent.create_turn(
messages=[
UserMessage(
role="user",
content=(
""
)
)
],
stream=True,
)

print("\n--- Agent response (stream) ---")
for chunk in turn:
if chunk.event_type == "assistant_message_delta":
print(chunk.delta, end="", flush=True)
print("\n--- End of stream ---")

ERROR:

INFO:httpx:HTTP Request: GET http://localhost:8321/v1/models "HTTP/1.1 426 Upgrade Required"
Traceback (most recent call last):
File "/Users/macbookair-unifynd/llama-stack-working/ollama-stack-examples/task.py", line 39, in
for m in client.models.list()
~~~~~~~~~~~~~~~~~~^^
File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages/llama_stack_client/resources/models/models.py", line 112, in list
return self._get(
~~~~~~~~~^
"/v1/models",
^^^^^^^^^^^^^
...<7 lines>...
cast_to=cast(Type[ModelListResponse], DataWrapper[ModelListResponse]),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages/llama_stack_client/_base_client.py", line 1198, in get
return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages/llama_stack_client/_base_client.py", line 1054, in request
raise self._make_status_error_from_response(err.response) from None
llama_stack_client.APIStatusError: Error code: 426 - {'error': {'message': 'Client version 0.3.0 is not compatible with server version 0.2.9. Please update your client.'}}

[mattf]

this appears to be py3.13 and mismatched versions of llama-stack-client and llama-stack.

[mattf]

@mansiUnifynd unfortunately i'm not sure how to help. the problematic environment has been py3.10, py3.12, py.3.13 with various combinations of old and new llama stack and llama stack client installed.

[mansiUnifynd]

No worries , I appreciate the clarification. I’ll continue investigating the issue on my end and will try isolating it across the different Python environments and llama stack versions. I’ll share an update once I have more concrete findings. Thanks a lot!