Skip to content

Add partner model support to vertexai providerΒ #5172

@major

Description

@major

πŸš€ Describe the new functionality needed

The recent overhaul of the remote::vertexai provider opens the door for consuming partner models, such as those from Anthropic and Mistral, via the Vertex AI APIs. Most of the existing code can be re-used, but there is some additional conversion necessary since we can't use the regular generate calls that we can with Gemini models.

πŸ’‘ Why is this needed? What if we don't build it?

This allows users to consume all of the built-in models in Vertex AI as well as the Anthropic/Mistral models. If we don't add this, then users can't use those partner models.

Other thoughts

I'm willing to do the work to build/test this.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions