Skip to content

Commit 6f0a7a8

Browse files
yuzisunnacx
andauthored
docs: update doc for extension fields (#1590)
**Description** Explained how the unified thinking field works across providers and documented automatic translation: GCP Vertex AI (Gemini) → generationConfig.thinkingConfig, while GCP Anthropic and AWS Bedrock use it directly. Update extension field usage for both vendor specific and unified fields. --------- Signed-off-by: Dan Sun <[email protected]> Co-authored-by: Ignasi Barrera <[email protected]>
1 parent 4e7d496 commit 6f0a7a8

File tree

1 file changed

+57
-16
lines changed

1 file changed

+57
-16
lines changed
Lines changed: 57 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,29 +1,43 @@
11
---
22
id: vendor-specific-fields
3-
title: Vendor-Specific Fields
3+
title: Extension Fields
44
---
55

6-
# Vendor-Specific Fields
6+
# Extension Fields
77

8-
The AI Gateway supports vendor-specific fields that allow you to specify backend-specific parameters directly as inline fields in your OpenAI-compatible requests. These fields are applied during the translation process to the target backend's native API format.
8+
The AI Gateway supports extension fields that allow you to specify unified or backend-specific parameters directly as inline fields in your OpenAI-compatible requests. These fields are applied during the translation process to the target backend's native API format.
99

1010
## Overview
1111

12-
Vendor-specific fields enable you to:
12+
Extension fields enable you to:
1313

1414
- Use advanced backend-specific features not available in the OpenAI API
15+
- Use unified configuration fields that work across multiple providers not available in the OpenAI API
1516

16-
The vendor-specific fields are specified as inline fields in your OpenAI request and are applied after the standard OpenAI-to-backend translation.
17+
### Vendor Extension Fields
18+
19+
Vendor specific fields are specified as inline fields in your OpenAI request and are applied after the standard OpenAI-to-backend translation.
20+
21+
### Unified Extension Fields
22+
23+
For thinking/reasoning capabilities, you can use a unified `thinking` field that automatically translates to the correct backend-specific format:
24+
25+
- **GCP Vertex AI (Gemini)**: Translates to `generationConfig.thinkingConfig`
26+
- **GCP Anthropic**: Uses `thinking` field directly
27+
- **AWS Bedrock**: Uses `thinking` field directly
28+
29+
This unified approach allows you to write provider-agnostic requests while still leveraging thinking capabilities.
1730

1831
## Supported Backends
1932

20-
The following backends support vendor-specific fields:
33+
The following backends support extension fields:
2134

2235
### GCP Vertex AI (Gemini)
2336

2437
- **API Schema Name**: `GCPVertexAI`
2538
- **Supported Fields**:
26-
- `generationConfig.thinkingConfig`: Configure thinking process for reasoning models. [Gemini Docs](https://cloud.google.com/vertex-ai/docs/reference/rest/v1/GenerationConfig#ThinkingConfig)
39+
- `safetySettings`: Configure the safety settings for gemini models that translates to `SafetySetting`. [Gemini Docs](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/multimodal/configure-safety-filters)
40+
- `thinking`: Configure thinking process for reasoning models that automatically translates to `generationConfig.thinkingConfig`. [Gemini Docs](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/thinking)
2741

2842
### GCP Anthropic
2943

@@ -39,11 +53,15 @@ The following backends support vendor-specific fields:
3953

4054
## Usage
4155

42-
Add vendor-specific fields directly as inline fields in your OpenAI request:
56+
Add extension fields directly as inline fields in your OpenAI request:
57+
58+
### Using Unified Thinking Configuration
59+
60+
The simplest way to enable thinking capabilities across all providers is to use the unified `thinking` field:
4361

4462
```json
4563
{
46-
"model": "gemini-1.5-pro",
64+
"model": "gemini-2.5-pro",
4765
"messages": [
4866
{
4967
"role": "user",
@@ -54,21 +72,44 @@ Add vendor-specific fields directly as inline fields in your OpenAI request:
5472
"max_tokens": 2000,
5573
"thinking": {
5674
"type": "enabled",
57-
"budget_tokens": 1000
58-
},
59-
"generationConfig": {
60-
"thinkingConfig": {
61-
"includeThoughts": true,
62-
"thinkingBudget": 1000
63-
}
75+
"budget_tokens": 1000,
76+
"includeThoughts": true
6477
}
6578
}
6679
```
6780

81+
This configuration will work with any provider that supports thinking, automatically translating to the correct backend format.
82+
83+
### Using Provider-Specific Fields
84+
85+
For more fine-grained control or provider-specific features, you can use the vendor-specific fields like `safetySettings` for gemini models:
86+
87+
```json
88+
{
89+
"model": "gemini-1.5-pro",
90+
"messages": [
91+
{
92+
"role": "user",
93+
"content": "Explain quantum computing and show me a simple code example."
94+
}
95+
],
96+
"temperature": 0.7,
97+
"max_tokens": 2000,
98+
"safetySettings": [
99+
{
100+
"category": "HARM_CATEGORY_HARASSMENT",
101+
"threshold": "BLOCK_ONLY_HIGH"
102+
}
103+
]
104+
}
105+
```
106+
68107
### Field Conflicts
69108

70109
Vendor fields override translated fields when conflicts occur.
71110

111+
When using unified thinking configuration, the `thinking` field takes precedence over any provider-specific thinking configurations.
112+
72113
### Unsupported Fields/Backends
73114

74115
Fields and Backends other than specified in [Supported Backends](#supported-backends) will be ignored.

0 commit comments

Comments
 (0)