Why does claude-3-5-sonnet-latest not work as a model name?

[rcdailey]

In my config.yaml, I'd like to register a model under model_list like so:

model_list:
  - model_name: claude-3.5
    litellm_params:
      model: claude-3-5-sonnet-latest
      api_key: os.environ/ANTHROPIC_API_KEY

However this results in an error:

utils.py:4437 - Error getting model info: This model isn't mapped yet. Add it here - https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json

And sure enough when I check that file, it isn't there. I don't like having to always update my model name to point to the newest version each time. Is there a reason the -latest versions can't be registered?

[krrishdholakia]

utils.py:4437 - Error getting model info: This model isn't mapped yet. Add it here

hey @rcdailey how are you triggering this error?

I believe anthropic returns the model version name in the response, so the -latest should have no impact on cost tracking

[rcdailey]

I'm triggering the error with the above config.yaml. Did I leave out some important details?

It seems like it's validating that the model name I specify in the configuration exists in that linked JSON file, which in this case it does not.

I'm not sure what cost tracking is or what that has to do with this.

[krrishdholakia]

it should work if you add anthropic/ -> anthropic/claude-3-5-sonnet-latest

we're missing the -latest tags in our model cost map which we use to figure out what this name means

adding now

[rcdailey]

If the anthropic/ prefix is required, then why is it working if I do model: claude-3-5-sonnet-20241022? None of the examples I found show that the anthropic prefix is required. Not to say that you're wrong necessarily, but from my perspective it is confusing. If you could point me to documentation that explains the behavior with and without the prefix, that would be educational for me.

Secondly, why do you call it "cost tracking" instead of "model mapping" or "model list"? Is it that the mapping is technically for cost tracking purposes, which is why you word it that way, even if I'm not using it in the context of tracking cost?

Also, thank you very much for the fixes! I was able to verify that this is working:

model_list:
  - model_name: claude-3.5
    litellm_params:
      model: anthropic/claude-3-5-sonnet-latest
      api_key: os.environ/ANTHROPIC_API_KEY

  - model_name: claude-3.7
    litellm_params:
      model: anthropic/claude-3-7-sonnet-latest
      api_key: os.environ/ANTHROPIC_API_KEY

[krrishdholakia]

it works because claude-3-5-sonnet-20241022 is here -

litellm/model_prices_and_context_window.json

Line 2913 in f9cee4c

"claude-3-5-sonnet-20241022": {

[rcdailey]

And where does it look when I use the prefix?

[krrishdholakia]

Fixed - #8816

thanks for raising this @rcdailey