You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
**Description**
Explained how the unified thinking field works across providers and
documented automatic translation: GCP Vertex AI (Gemini) →
generationConfig.thinkingConfig, while GCP Anthropic and AWS Bedrock use
it directly. Update extension field usage for both vendor specific and
unified fields.
---------
Signed-off-by: Dan Sun <[email protected]>
Co-authored-by: Ignasi Barrera <[email protected]>
The AI Gateway supports vendor-specific fields that allow you to specify backend-specific parameters directly as inline fields in your OpenAI-compatible requests. These fields are applied during the translation process to the target backend's native API format.
8
+
The AI Gateway supports extension fields that allow you to specify unified or backend-specific parameters directly as inline fields in your OpenAI-compatible requests. These fields are applied during the translation process to the target backend's native API format.
9
9
10
10
## Overview
11
11
12
-
Vendor-specific fields enable you to:
12
+
Extension fields enable you to:
13
13
14
14
- Use advanced backend-specific features not available in the OpenAI API
15
+
- Use unified configuration fields that work across multiple providers not available in the OpenAI API
15
16
16
-
The vendor-specific fields are specified as inline fields in your OpenAI request and are applied after the standard OpenAI-to-backend translation.
17
+
### Vendor Extension Fields
18
+
19
+
Vendor specific fields are specified as inline fields in your OpenAI request and are applied after the standard OpenAI-to-backend translation.
20
+
21
+
### Unified Extension Fields
22
+
23
+
For thinking/reasoning capabilities, you can use a unified `thinking` field that automatically translates to the correct backend-specific format:
24
+
25
+
-**GCP Vertex AI (Gemini)**: Translates to `generationConfig.thinkingConfig`
26
+
-**GCP Anthropic**: Uses `thinking` field directly
27
+
-**AWS Bedrock**: Uses `thinking` field directly
28
+
29
+
This unified approach allows you to write provider-agnostic requests while still leveraging thinking capabilities.
17
30
18
31
## Supported Backends
19
32
20
-
The following backends support vendor-specific fields:
33
+
The following backends support extension fields:
21
34
22
35
### GCP Vertex AI (Gemini)
23
36
24
37
-**API Schema Name**: `GCPVertexAI`
25
38
-**Supported Fields**:
26
-
-`generationConfig.thinkingConfig`: Configure thinking process for reasoning models. [Gemini Docs](https://cloud.google.com/vertex-ai/docs/reference/rest/v1/GenerationConfig#ThinkingConfig)
39
+
-`safetySettings`: Configure the safety settings for gemini models that translates to `SafetySetting`. [Gemini Docs](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/multimodal/configure-safety-filters)
40
+
-`thinking`: Configure thinking process for reasoning models that automatically translates to `generationConfig.thinkingConfig`. [Gemini Docs](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/thinking)
27
41
28
42
### GCP Anthropic
29
43
@@ -39,11 +53,15 @@ The following backends support vendor-specific fields:
39
53
40
54
## Usage
41
55
42
-
Add vendor-specific fields directly as inline fields in your OpenAI request:
56
+
Add extension fields directly as inline fields in your OpenAI request:
57
+
58
+
### Using Unified Thinking Configuration
59
+
60
+
The simplest way to enable thinking capabilities across all providers is to use the unified `thinking` field:
43
61
44
62
```json
45
63
{
46
-
"model": "gemini-1.5-pro",
64
+
"model": "gemini-2.5-pro",
47
65
"messages": [
48
66
{
49
67
"role": "user",
@@ -54,21 +72,44 @@ Add vendor-specific fields directly as inline fields in your OpenAI request:
54
72
"max_tokens": 2000,
55
73
"thinking": {
56
74
"type": "enabled",
57
-
"budget_tokens": 1000
58
-
},
59
-
"generationConfig": {
60
-
"thinkingConfig": {
61
-
"includeThoughts": true,
62
-
"thinkingBudget": 1000
63
-
}
75
+
"budget_tokens": 1000,
76
+
"includeThoughts": true
64
77
}
65
78
}
66
79
```
67
80
81
+
This configuration will work with any provider that supports thinking, automatically translating to the correct backend format.
82
+
83
+
### Using Provider-Specific Fields
84
+
85
+
For more fine-grained control or provider-specific features, you can use the vendor-specific fields like `safetySettings` for gemini models:
86
+
87
+
```json
88
+
{
89
+
"model": "gemini-1.5-pro",
90
+
"messages": [
91
+
{
92
+
"role": "user",
93
+
"content": "Explain quantum computing and show me a simple code example."
94
+
}
95
+
],
96
+
"temperature": 0.7,
97
+
"max_tokens": 2000,
98
+
"safetySettings": [
99
+
{
100
+
"category": "HARM_CATEGORY_HARASSMENT",
101
+
"threshold": "BLOCK_ONLY_HIGH"
102
+
}
103
+
]
104
+
}
105
+
```
106
+
68
107
### Field Conflicts
69
108
70
109
Vendor fields override translated fields when conflicts occur.
71
110
111
+
When using unified thinking configuration, the `thinking` field takes precedence over any provider-specific thinking configurations.
112
+
72
113
### Unsupported Fields/Backends
73
114
74
115
Fields and Backends other than specified in [Supported Backends](#supported-backends) will be ignored.
0 commit comments