Skip to content

[FEATURE] Add ai_gateway Terraform support for (non-external) Provisioned Throughput Foundation Model endpoints #5201

@stephanieylee

Description

@stephanieylee

Use-cases

Databricks provides additional controls (AI Gateway) for FM serving endpoints, and there is Terraform support for the ai_gateway block for external models, but not for built-in foundation models served by Provisioned Throughput endpoints.

Terraform should support built-in foundation models as well so that users can use IaC to stand up and govern endpoints for built-in and external FM endpoints.

Attempted Solutions

According to documentation, ai_gateway block is only for external models.

Proposal

Incorporate ai_gateway settings for built-in FM model configuration

References

#4029
https://docs.databricks.com/api/workspace/servingendpoints/putaigateway
https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/model_serving

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions