Skip to content

docs: BYO OpenAI-compatible provider #208

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
artberger merged 3 commits into from
Sep 10, 2025
Merged

docs: BYO OpenAI-compatible provider #208

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
44ed08f
BYO OpenAI-compatible provider
artberger Sep 9, 2025
a740363
testing updates
artberger Sep 10, 2025
612e1d0
Merge branch 'main' into adb-byo-model
artberger Sep 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion public/sitemap.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,14 @@

<url>
<loc>https://kagent.dev/docs/kagent/supported-providers/azure-openai</loc>
<lastmod>2025-09-10</lastmod>
<lastmod>2025-09-09</lastmod>
<changefreq>weekly</changefreq>
<priority>0.8</priority>
</url>

<url>
<loc>https://kagent.dev/docs/kagent/supported-providers/byo-openai</loc>
<lastmod>2025-09-09</lastmod>
<changefreq>weekly</changefreq>
<priority>0.8</priority>
</url>
Expand Down
57 changes: 57 additions & 0 deletions src/app/docs/kagent/supported-providers/byo-openai/page.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
---
title: "BYO OpenAI-compatible model"
pageOrder: 7
description: "Bring your own OpenAI-compatible model to kagent."
---

export const metadata = {
title: "BYO OpenAI-compatible model",
description: "Bring your own OpenAI-compatible model to kagent.",
author: "kagent.dev"
};

You can bring your own model from an [OpenAI API-compatible](https://platform.openai.com/docs/api-reference/introduction) LLM provider. The example integrates with [Cohere AI](https://cohere.com/).

1. Save your API key from the OpenAI-compatible provider as an environment variable. For example, navigate to the [Cohere AI dashboard](https://dashboard.cohere.com/api-keys).

```bash
export PROVIDER_API_KEY=Dgs...
```

2. Create a Kubernetes secret that stores your API key. Make sure to create the secret in the same namespace as you plan to create your agent, such as `kagent`.

```bash
kubectl create secret generic kagent-my-provider -n kagent --from-literal PROVIDER_API_KEY=$PROVIDER_API_KEY
```

3. Create a ModelConfig resource.

```yaml
kubectl apply -f - <<EOF
apiVersion: kagent.dev/v1alpha2
kind: ModelConfig
metadata:
name: my-provider-config
namespace: kagent
spec:
apiKeySecret: kagent-my-provider
apiKeySecretKey: PROVIDER_API_KEY
model: command-a-03-2025
provider: OpenAI
openAI:
baseUrl: "https://api.cohere.ai/compatibility/v1"
EOF
```

Review the following table to understand this configuration. For more information, see the [API docs](https://kagent.dev/docs/kagent/resources/api-ref#openaiconfig).

| Setting | Description |
| --- | --- |
| `apiKeySecret` | The name of the Kubernetes secret that stores your API key. |
| `apiKeySecretKey` | The key in the secret that stores your API key. |
| `model` | The OpenAI API-compatible model to use. For more information about the model, consult your LLM provider's documentation. For example, you might use `command-a-03-2025` for [Cohere AI](https://docs.cohere.com/v1/docs/models). |
| `provider` | To use an OpenAI API-compatible model, set the provider to `OpenAI`. |
| `openAI` | Additional provider details. For available settings, consult your LLM provider's documentation. At the least, you must configure the `baseUrl` setting to point to the endpoint of your LLM provider. |
| `baseUrl` | The base URL of your LLM provider. Note that the LLM provider might have a special base URL for OpenAI compatibility, such as `"https://api.cohere.ai/compatibility/v1"` for [Cohere AI](https://cohere.com/). |

Good job! You added a model to kagent. Next, you can [create or update an agent](https://kagent.dev/docs/kagent/getting-started/first-agent) to use this model.
5 changes: 5 additions & 0 deletions src/config/navigation.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,11 @@
"title": "OpenAI",
"href": "/docs/kagent/supported-providers/openai",
"description": "Learn how to configure OpenAI models in kagent."
},
{
"title": "BYO OpenAI-compatible model",
"href": "/docs/kagent/supported-providers/byo-openai",
"description": "Bring your own OpenAI-compatible model to kagent."
}
]
},
Expand Down