Skip to content

Auto-determine the anthropic model context window size #342

@bhouston

Description

@bhouston

There is support within the Anthropic SDK for getting the model context window size. Right now the anthropic LLM provider has all the model sizes hard coded. Here is an example of how it can be determined:

    const response = await this.anthropicClient.models.list();
      const model = response.data.find(m => m.id === modelName);
      
      if (!model) {
        return null;
      }

      return {
        name: model.id,
        contextWindow: model.context_window,
        provider: 'anthropic',
      };

We should replace the existing hard coded method with this proper detection method.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions