Skip to content

Commit 326df23

Browse files
TarquinenTarquinen
authored
Merge pull request #64 from Tarquinen/feat/bedrock-support
feat: add AWS Bedrock Converse API support and provider docs
2 parents 9ebf1c5 + 963d6bc commit 326df23

File tree

13 files changed

+2125
-4
lines changed

13 files changed

+2125
-4
lines changed

‎.gitignore‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -31,5 +31,3 @@ Thumbs.db
3131
# Tests (local development only)
3232
tests/
3333

34-
# Development notes
35-
notes/

‎.npmignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,8 @@ bun.lock
1010

1111
# Documentation
1212
ANALYSIS.md
13+
docs/
14+
notes/
1315

1416
# Source files (since we're shipping dist/)
1517
index.ts

‎docs/providers/README.md‎

Lines changed: 339 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,339 @@
1+
# Provider API Formats Reference
2+
3+
This directory contains documentation for each AI provider's API format, designed to help the context pruning plugin implement provider-specific logic.
4+
5+
## Sources
6+
7+
All information in these docs was gathered from:
8+
9+
### Primary Sources
10+
11+
| Source | Location | Description |
12+
|--------|----------|-------------|
13+
| **Vercel AI SDK** | https://github.com/vercel/ai | Provider conversion logic in `packages/{provider}/src/` |
14+
| **OpenCode Source** | `/packages/opencode/src/provider/` | Custom transforms and provider loading |
15+
| **models.dev API** | https://models.dev/api.json | Authoritative provider list with npm packages |
16+
17+
### Key AI SDK Files
18+
19+
| Provider | Conversion File |
20+
|----------|-----------------|
21+
| OpenAI | `packages/openai/src/chat/openai-chat-language-model.ts`, `packages/openai/src/responses/openai-responses-language-model.ts` |
22+
| OpenAI-Compatible | `packages/openai-compatible/src/chat/openai-compatible-chat-language-model.ts` |
23+
| Anthropic | `packages/anthropic/src/convert-to-anthropic-messages-prompt.ts`, `packages/anthropic/src/anthropic-messages-language-model.ts` |
24+
| Google | `packages/google/src/convert-to-google-generative-ai-messages.ts`, `packages/google/src/google-generative-ai-language-model.ts` |
25+
| AWS Bedrock | `packages/amazon-bedrock/src/convert-to-bedrock-chat-messages.ts`, `packages/amazon-bedrock/src/bedrock-chat-language-model.ts` |
26+
| Mistral | `packages/mistral/src/convert-to-mistral-chat-messages.ts`, `packages/mistral/src/mistral-chat-language-model.ts` |
27+
| Cohere | `packages/cohere/src/convert-to-cohere-chat-prompt.ts`, `packages/cohere/src/cohere-chat-language-model.ts` |
28+
29+
### OpenCode Custom Transform Files
30+
31+
| File | Purpose |
32+
|------|---------|
33+
| `src/provider/transform.ts` | Provider-specific message normalization, caching hints, schema transforms |
34+
| `src/provider/provider.ts` | Provider loading, custom loaders, SDK instantiation |
35+
| `src/provider/models.ts` | Model database schema, models.dev integration |
36+
| `src/session/message-v2.ts` | Internal message structure, `toModelMessage()` conversion |
37+
38+
### Official API Documentation
39+
40+
| Provider | Documentation URL |
41+
|----------|-------------------|
42+
| OpenAI | https://platform.openai.com/docs/api-reference |
43+
| Anthropic | https://docs.anthropic.com/en/api |
44+
| Google Gemini | https://ai.google.dev/api/rest |
45+
| AWS Bedrock | https://docs.aws.amazon.com/bedrock/latest/APIReference/ |
46+
| Mistral | https://docs.mistral.ai/api/ |
47+
| Cohere | https://docs.cohere.com/reference/chat |
48+
49+
---
50+
51+
## Format Categories
52+
53+
Providers fall into several format categories based on their API structure:
54+
55+
### 1. OpenAI Chat Completions Format
56+
**Most common format - used by ~60 providers**
57+
58+
Key identifiers:
59+
- `body.messages[]` array
60+
- Tool results: `role: "tool"`, `tool_call_id`
61+
- System in messages array
62+
63+
Providers: openai, together, deepseek, groq, fireworks, hyperbolic, novita, cerebras, sambanova, perplexity, openrouter, and most others
64+
65+
### 2. OpenAI Responses Format (newer)
66+
**Used by OpenAI GPT models via responses API**
67+
68+
Key identifiers:
69+
- `body.input[]` array
70+
- Tool results: `type: "function_call_output"`, `call_id`
71+
72+
Providers: openai (responses endpoint), azure (responses endpoint)
73+
74+
### 3. Anthropic Format
75+
**Distinct format with cache control**
76+
77+
Key identifiers:
78+
- `body.messages[]` but tool results in user messages
79+
- Tool results: `type: "tool_result"`, `tool_use_id`
80+
- Top-level `system` array
81+
- `cache_control` support
82+
83+
Providers: anthropic
84+
85+
### 4. Google Gemini Format
86+
**Position-based tool correlation**
87+
88+
Key identifiers:
89+
- `body.contents[]` array
90+
- Tool results: `functionResponse` parts (no IDs!)
91+
- Roles: `user`/`model` only
92+
- Top-level `systemInstruction`
93+
94+
Providers: google, google-vertex
95+
96+
### 5. AWS Bedrock Format
97+
**Converse API with cache points**
98+
99+
Key identifiers:
100+
- Top-level `system` array
101+
- Tool results: `toolResult` blocks with `toolUseId`
102+
- `cachePoint` blocks
103+
104+
Providers: amazon-bedrock
105+
106+
### 6. Mistral Format (OpenAI-like with quirks)
107+
**Strict ID requirements**
108+
109+
Key identifiers:
110+
- OpenAI-like but 9-char alphanumeric tool IDs required
111+
- User content always array
112+
113+
Providers: mistral
114+
115+
### 7. Cohere Format
116+
**RAG-native with citations**
117+
118+
Key identifiers:
119+
- Uses `p`/`k` instead of `top_p`/`top_k`
120+
- Uppercase tool choice values
121+
- `documents` array for RAG
122+
123+
Providers: cohere
124+
125+
## Quick Reference: Thinking/Reasoning
126+
127+
| Format | Request Config | Response Structure | Encrypted? | Signature? |
128+
|--------|---------------|-------------------|------------|------------|
129+
| OpenAI Responses | `reasoning: {effort, summary}` | `{type: "reasoning", encrypted_content, summary}` | Yes | No |
130+
| Anthropic | `thinking: {type, budget_tokens}` | `{type: "thinking", thinking, signature}` | Partial* | Yes |
131+
| Google Gemini | `thinkingConfig: {thinkingBudget}` | `{text, thought: true, thoughtSignature}` | No | Optional |
132+
| AWS Bedrock | `additionalModelRequestFields.thinking` | `{reasoningContent: {reasoningText/redactedReasoning}}` | Partial* | Yes |
133+
| Mistral | N/A (model decides) | `{type: "thinking", thinking: [{type: "text", text}]}` | No | No |
134+
| Cohere | `thinking: {type, token_budget}` | `{type: "thinking", thinking: "..."}` | No | No |
135+
136+
*Partial = has both visible (`thinking`/`reasoningText`) and redacted (`redacted_thinking`/`redactedReasoning`) variants
137+
138+
**Key differences:**
139+
- **OpenAI**: Reasoning is always encrypted; only summary is readable
140+
- **Anthropic/Bedrock**: Can have visible thinking with signature, or redacted thinking
141+
- **Gemini**: Thinking is a text part with `thought: true` flag
142+
- **Mistral**: Thinking is nested array of text parts
143+
- **Cohere**: Thinking is plain string
144+
145+
**SDK normalization**: All formats are converted to `{type: "reasoning", text: "..."}` by the AI SDK
146+
147+
## Quick Reference: Tool Call ID Fields
148+
149+
| Format | Tool Call ID Field | Tool Result ID Field |
150+
|--------|-------------------|---------------------|
151+
| OpenAI Chat | `tool_calls[].id` | `tool_call_id` |
152+
| OpenAI Responses | `call_id` | `call_id` |
153+
| Anthropic | `tool_use.id` | `tool_use_id` |
154+
| Gemini | **NONE (position-based)** | **NONE** |
155+
| Bedrock | `toolUse.toolUseId` | `toolResult.toolUseId` |
156+
| Mistral | `tool_calls[].id` (9-char) | `tool_call_id` |
157+
| Cohere | `tool_calls[].id` | `tool_call_id` |
158+
159+
## Detection Strategy
160+
161+
To detect which format a request uses:
162+
163+
```typescript
164+
function detectFormat(body: unknown): string {
165+
if (body.input && Array.isArray(body.input)) return 'openai-responses'
166+
if (body.contents && Array.isArray(body.contents)) return 'gemini'
167+
if (body.system && Array.isArray(body.system) && body.inferenceConfig) return 'bedrock'
168+
if (body.messages) {
169+
// Check first message structure for Anthropic vs OpenAI
170+
const msg = body.messages[0]
171+
if (msg?.content?.[0]?.type === 'tool_result') return 'anthropic'
172+
if (msg?.content?.[0]?.tool_use_id) return 'anthropic'
173+
}
174+
return 'openai-chat' // Default
175+
}
176+
```
177+
178+
## Files
179+
180+
- [openai.md](./openai.md) - OpenAI Chat Completions & Responses API
181+
- [anthropic.md](./anthropic.md) - Anthropic Messages API
182+
- [google-gemini.md](./google-gemini.md) - Google Generative AI (Gemini)
183+
- [aws-bedrock.md](./aws-bedrock.md) - AWS Bedrock Converse API
184+
- [mistral.md](./mistral.md) - Mistral API
185+
- [cohere.md](./cohere.md) - Cohere Chat API
186+
- [openai-compatible.md](./openai-compatible.md) - OpenAI-compatible providers
187+
188+
## Context Pruning Universal Rules
189+
190+
1. **Tool call/result pairing**: Always prune tool calls and their results together
191+
2. **Message alternation**: Most APIs expect alternating user/assistant messages
192+
3. **System preservation**: System messages typically should not be pruned
193+
4. **ID correlation**: Maintain ID relationships when pruning (except Gemini which is position-based)
194+
5. **Cache markers**: Consider preserving cache control markers when present
195+
196+
---
197+
198+
## Complete Provider List (models.dev)
199+
200+
Every provider from models.dev and its API format:
201+
202+
### OpenAI Chat Format (43 providers)
203+
*Uses `@ai-sdk/openai-compatible` - standard OpenAI messages format*
204+
205+
| Provider ID | Name | Notes |
206+
|-------------|------|-------|
207+
| `agentrouter` | AgentRouter | |
208+
| `alibaba` | Alibaba | |
209+
| `alibaba-cn` | Alibaba (China) | |
210+
| `bailing` | Bailing | |
211+
| `baseten` | Baseten | |
212+
| `chutes` | Chutes | |
213+
| `cortecs` | Cortecs | |
214+
| `deepseek` | DeepSeek | Reasoning models (R1) |
215+
| `fastrouter` | FastRouter | |
216+
| `fireworks-ai` | Fireworks AI | |
217+
| `github-copilot` | GitHub Copilot | |
218+
| `github-models` | GitHub Models | |
219+
| `huggingface` | Hugging Face | |
220+
| `iflowcn` | iFlow | |
221+
| `inception` | Inception | |
222+
| `inference` | Inference | |
223+
| `io-net` | IO.NET | |
224+
| `llama` | Llama | |
225+
| `lmstudio` | LMStudio | Local inference |
226+
| `lucidquery` | LucidQuery AI | |
227+
| `modelscope` | ModelScope | |
228+
| `moonshotai` | Moonshot AI | |
229+
| `moonshotai-cn` | Moonshot AI (China) | |
230+
| `morph` | Morph | |
231+
| `nebius` | Nebius Token Factory | |
232+
| `nvidia` | Nvidia | |
233+
| `opencode` | OpenCode Zen | |
234+
| `openrouter` | OpenRouter | Meta-provider, cache support |
235+
| `ovhcloud` | OVHcloud AI Endpoints | |
236+
| `poe` | Poe | |
237+
| `requesty` | Requesty | |
238+
| `scaleway` | Scaleway | |
239+
| `siliconflow` | SiliconFlow | |
240+
| `submodel` | submodel | |
241+
| `synthetic` | Synthetic | |
242+
| `upstage` | Upstage | |
243+
| `venice` | Venice AI | |
244+
| `vultr` | Vultr | |
245+
| `wandb` | Weights & Biases | |
246+
| `zai` | Z.AI | |
247+
| `zai-coding-plan` | Z.AI Coding Plan | |
248+
| `zenmux` | ZenMux | |
249+
| `zhipuai` | Zhipu AI | |
250+
| `zhipuai-coding-plan` | Zhipu AI Coding Plan | |
251+
252+
### OpenAI Native Format (1 provider)
253+
*Uses `@ai-sdk/openai` - supports both Chat Completions and Responses API*
254+
255+
| Provider ID | Name | Notes |
256+
|-------------|------|-------|
257+
| `openai` | OpenAI | Responses API for GPT-4.1+ |
258+
259+
### Azure Format (2 providers)
260+
*Uses `@ai-sdk/azure` - OpenAI format with Azure auth*
261+
262+
| Provider ID | Name | Notes |
263+
|-------------|------|-------|
264+
| `azure` | Azure | Supports Responses API |
265+
| `azure-cognitive-services` | Azure Cognitive Services | |
266+
267+
### Anthropic Format (4 providers)
268+
*Uses `@ai-sdk/anthropic` - distinct message format with cache control*
269+
270+
| Provider ID | Name | Notes |
271+
|-------------|------|-------|
272+
| `anthropic` | Anthropic | Native Anthropic API |
273+
| `kimi-for-coding` | Kimi For Coding | Uses Anthropic format |
274+
| `minimax` | MiniMax | Uses Anthropic format |
275+
| `minimax-cn` | MiniMax (China) | Uses Anthropic format |
276+
277+
### Google Gemini Format (3 providers)
278+
*Uses `@ai-sdk/google` or `@ai-sdk/google-vertex` - POSITION-BASED tool correlation*
279+
280+
| Provider ID | Name | Notes |
281+
|-------------|------|-------|
282+
| `google` | Google | Native Gemini API |
283+
| `google-vertex` | Vertex | Google Cloud Vertex AI |
284+
| `google-vertex-anthropic` | Vertex (Anthropic) | Claude via Vertex |
285+
286+
### AWS Bedrock Format (1 provider)
287+
*Uses `@ai-sdk/amazon-bedrock` - Converse API with cachePoint*
288+
289+
| Provider ID | Name | Notes |
290+
|-------------|------|-------|
291+
| `amazon-bedrock` | Amazon Bedrock | Multi-model, cachePoint support |
292+
293+
### Mistral Format (1 provider)
294+
*Uses `@ai-sdk/mistral` - requires 9-char alphanumeric tool IDs*
295+
296+
| Provider ID | Name | Notes |
297+
|-------------|------|-------|
298+
| `mistral` | Mistral | Strict tool ID format |
299+
300+
### Cohere Format (1 provider)
301+
*Uses `@ai-sdk/cohere` - RAG-native with citations*
302+
303+
| Provider ID | Name | Notes |
304+
|-------------|------|-------|
305+
| `cohere` | Cohere | Uses `p`/`k`, uppercase tool choice |
306+
307+
### Specialized SDK Providers (13 providers)
308+
*Use provider-specific SDKs but follow OpenAI-like format*
309+
310+
| Provider ID | Name | SDK | Format |
311+
|-------------|------|-----|--------|
312+
| `cerebras` | Cerebras | `@ai-sdk/cerebras` | OpenAI-like |
313+
| `deepinfra` | Deep Infra | `@ai-sdk/deepinfra` | OpenAI-like |
314+
| `groq` | Groq | `@ai-sdk/groq` | OpenAI-like |
315+
| `perplexity` | Perplexity | `@ai-sdk/perplexity` | OpenAI-like |
316+
| `togetherai` | Together AI | `@ai-sdk/togetherai` | OpenAI-like |
317+
| `xai` | xAI | `@ai-sdk/xai` | OpenAI-like |
318+
| `vercel` | Vercel AI Gateway | `@ai-sdk/gateway` | OpenAI-like |
319+
| `v0` | v0 | `@ai-sdk/vercel` | OpenAI-like |
320+
| `cloudflare-workers-ai` | Cloudflare Workers AI | `workers-ai-provider` | OpenAI-like |
321+
| `ollama-cloud` | Ollama Cloud | `ai-sdk-ollama` | OpenAI-like |
322+
| `aihubmix` | AIHubMix | `@aihubmix/ai-sdk-provider` | OpenAI-like |
323+
| `sap-ai-core` | SAP AI Core | `@mymediset/sap-ai-provider` | OpenAI-like |
324+
325+
---
326+
327+
## Format Summary
328+
329+
| Format | Provider Count | Tool ID Field | Key Identifier |
330+
|--------|---------------|---------------|----------------|
331+
| OpenAI Chat | 56 | `tool_call_id` | `body.messages[]` |
332+
| OpenAI Responses | 2 | `call_id` | `body.input[]` |
333+
| Anthropic | 4 | `tool_use_id` | `tool_result` in user msg |
334+
| Google Gemini | 3 | **NONE** | `body.contents[]` |
335+
| AWS Bedrock | 1 | `toolUseId` | `body.inferenceConfig` |
336+
| Mistral | 1 | `tool_call_id` (9-char) | Check provider ID |
337+
| Cohere | 1 | `tool_call_id` | Check provider ID |
338+
339+
**Total: 69 providers**

0 commit comments

Comments
 (0)