Skip to content

Commit f7742a2

Browse files
MaedahBatoolMaedahBatool
authored
Docs for Claude 3.7 support (#1008)
- https://linear.app/sourcegraph/issue/CODY-5155/claude-37-sonnet-ga-gtm-enablement
1 parent c8fc405 commit f7742a2

File tree

4 files changed

+83
-43
lines changed

4 files changed

+83
-43
lines changed

‎docs/cody/capabilities/supported-models.mdx‎

Lines changed: 11 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -8,19 +8,20 @@ Cody supports a variety of cutting-edge large language models for use in chat an
88

99
| **Provider** | **Model** | **Free** | **Pro** | **Enterprise** | | | | |
1010
| :------------ | :-------------------------------------------------------------------------------------------------------------------------------------------- | :----------- | :----------- | :------------- | --- | --- | --- | --- |
11-
| OpenAI | [gpt-4 turbo](https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo#:~:text=TRAINING%20DATA-,gpt%2D4%2D0125%2Dpreview,-New%20GPT%2D4) | - | ✅ | ✅ | | | | |
12-
| OpenAI | [gpt-4o](https://platform.openai.com/docs/models#gpt-4o) | - | ✅ | ✅ | | | | |
13-
| OpenAI | [gpt-4o-mini](https://platform.openai.com/docs/models#gpt-4o-mini) | ✅ | ✅ | ✅ | | | | |
11+
| OpenAI | [GPT-4 Turbo](https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo#:~:text=TRAINING%20DATA-,gpt%2D4%2D0125%2Dpreview,-New%20GPT%2D4) | - | ✅ | ✅ | | | | |
12+
| OpenAI | [GPT-4o](https://platform.openai.com/docs/models#gpt-4o) | - | ✅ | ✅ | | | | |
13+
| OpenAI | [GPT-4o-mini](https://platform.openai.com/docs/models#gpt-4o-mini) | ✅ | ✅ | ✅ | | | | |
1414
| OpenAI | [o3-mini-medium](https://openai.com/index/openai-o3-mini/) (experimental) | ✅ | ✅ | ✅ | | | | |
1515
| OpenAI | [o3-mini-high](https://openai.com/index/openai-o3-mini/) (experimental) | - | - | ✅ | | | | |
1616
| OpenAI | [o1](https://platform.openai.com/docs/models#o1) | - | ✅ | ✅ | | | | |
17-
| Anthropic | [claude-3.5 Haiku](https://docs.anthropic.com/claude/docs/models-overview#model-comparison) | ✅ | ✅ | ✅ | | | | |
18-
| Anthropic | [claude-3.5 Sonnet](https://docs.anthropic.com/claude/docs/models-overview#model-comparison) | ✅ | ✅ | ✅ | | | | |
19-
| Google Gemini | [1.5 Pro](https://deepmind.google/technologies/gemini/pro/) | ✅ | ✅ | ✅ (beta) | | | | |
20-
| Google Gemini | [2.0 Flash](https://deepmind.google/technologies/gemini/flash/) | ✅ | ✅ | ✅ | | | | |
21-
| Google Gemini | [2.0 Flash-Lite Preview](https://deepmind.google/technologies/gemini/flash/) (experimental) | ✅ | ✅ | ✅ | | | | |
22-
23-
<Callout type="note">To use Claude 3 Sonnet models with Cody Enterprise, make sure you've upgraded your Sourcegraph instance to the latest version.</Callout>
17+
| Anthropic | [Claude 3.5 Haiku](https://docs.anthropic.com/claude/docs/models-overview#model-comparison) | ✅ | ✅ | ✅ | | | | |
18+
| Anthropic | [Claude 3.5 Sonnet](https://docs.anthropic.com/claude/docs/models-overview#model-comparison) | ✅ | ✅ | ✅ | | | | |
19+
| Anthropic | [Claude 3.7 Sonnet](https://docs.anthropic.com/claude/docs/models-overview#model-comparison) | - | ✅ | ✅ | | | | |
20+
| Google | [Gemini 1.5 Pro](https://deepmind.google/technologies/gemini/pro/) | ✅ | ✅ | ✅ (beta) | | | | |
21+
| Google | [Gemini 2.0 Flash](https://deepmind.google/technologies/gemini/flash/) | ✅ | ✅ | ✅ | | | | |
22+
| Google | [Gemini 2.0 Flash-Lite Preview](https://deepmind.google/technologies/gemini/flash/) (experimental) | ✅ | ✅ | ✅ | | | | |
23+
24+
<Callout type="note">To use Claude 3 Sonnet models with Cody Enterprise, make sure you've upgraded your Sourcegraph instance to the latest version. Claude 3.7 Sonnet with thinking is not supported for BYOK deployments.</Callout>
2425

2526
## Autocomplete
2627

‎docs/cody/core-concepts/token-limits.mdx‎

Lines changed: 35 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -13,53 +13,55 @@ Here's a detailed breakdown of the token limits by model:
1313

1414
<Tabs>
1515
<Tab title="Free">
16-
| **Model** | **Conversation Context** | **@-mention Context** | **Output** |
17-
| ------------------------------------ | ------------------------ | --------------------- | ---------- |
18-
| gpt-4o-mini | 7,000 | shared | 4,000 |
19-
| gpt-o3-mini-medium | 7,000 | shared | 4,000 |
20-
| claude-3.5 Haiku | 7,000 | shared | 4,000 |
21-
| **claude-3.5 Sonnet (New)** | **15,000** | **30,000** | **4,000** |
22-
| Google Gemini 1.5 Pro | 7,000 | shared | 4,000 |
23-
| Google Gemini 2.0 Flash | 7,000 | shared | 4,000 |
24-
| Google Gemini 2.0 Flash-Lite Preview | 7,000 | shared | 4,000 |
16+
| **Model** | **Conversation Context** | **@-mention Context** | **Output** |
17+
| ----------------------------- | ------------------------ | --------------------- | ---------- |
18+
| GPT 4o mini | 7,000 | shared | 4,000 |
19+
| GPT o3 mini medium | 7,000 | shared | 4,000 |
20+
| Claude 3.5 Haiku | 7,000 | shared | 4,000 |
21+
| **Claude 3.5 Sonnet (New)** | **15,000** | **30,000** | **4,000** |
22+
| Gemini 1.5 Pro | 7,000 | shared | 4,000 |
23+
| Gemini 2.0 Flash | 7,000 | shared | 4,000 |
24+
| Gemini 2.0 Flash-Lite Preview | 7,000 | shared | 4,000 |
2525

2626
</Tab>
2727

2828
<Tab title="Pro">
2929

3030
The Pro tier supports the token limits for the LLM models on Free tier, plus:
3131

32-
| **Model** | **Conversation Context** | **@-mention Context** | **Output** |
33-
| ------------------------------------ | ------------------------ | --------------------- | ---------- |
34-
| gpt-4o-mini | 7,000 | shared | 4,000 |
35-
| gpt-o3-mini-medium | 7,000 | shared | 4,000 |
36-
| gpt-4-turbo | 7,000 | shared | 4,000 |
37-
| gpt-4o | 7,000 | shared | 4,000 |
38-
| o1 | 7,000 | shared | 4,000 |
39-
| claude-3.5 Haiku | 7,000 | shared | 4,000 |
40-
| **claude-3.5 Sonnet (New)** | **15,000** | **30,000** | **4,000** |
41-
| **Google Gemini 1.5 Pro** | **15,000** | **30,000** | **4,000** |
42-
| Google Gemini 2.0 Flash | 7,000 | shared | 4,000 |
43-
| Google Gemini 2.0 Flash-Lite Preview | 7,000 | shared | 4,000 |
32+
| **Model** | **Conversation Context** | **@-mention Context** | **Output** |
33+
| ----------------------------- | ------------------------ | --------------------- | ---------- |
34+
| GPT 4o mini | 7,000 | shared | 4,000 |
35+
| GPT o3 mini medium | 7,000 | shared | 4,000 |
36+
| GPT 4 Turbo | 7,000 | shared | 4,000 |
37+
| GPT 4o | 7,000 | shared | 4,000 |
38+
| o1 | 7,000 | shared | 4,000 |
39+
| Claude 3.5 Haiku | 7,000 | shared | 4,000 |
40+
| **Claude 3.5 Sonnet (New)** | **15,000** | **30,000** | **4,000** |
41+
| Claude 3.7 Sonnet | 15,000 | 30,000 | 4,000 |
42+
| Gemini 1.5 Pro | 15,000 | 30,000 | 4,000 |
43+
| Gemini 2.0 Flash | 7,000 | shared | 4,000 |
44+
| Gemini 2.0 Flash-Lite Preview | 7,000 | shared | 4,000 |
4445

4546
</Tab>
4647

4748
<Tab title="Enterprise">
4849

4950
The Enterprise tier supports the token limits for the LLM models on Free and Pro tier, plus:
5051

51-
| **Model** | **Conversation Context** | **@-mention Context** | **Output** |
52-
| ------------------------------------ | ------------------------ | --------------------- | ---------- |
53-
| gpt-4o-mini | 7,000 | shared | 4,000 |
54-
| gpt-o3-mini-medium | 7,000 | shared | 4,000 |
55-
| gpt-4-turbo | 7,000 | shared | 4,000 |
56-
| gpt-4o | 7,000 | shared | 4,000 |
57-
| o1 | 7,000 | shared | 4,000 |
58-
| o3-mini-high | 7,000 | shared | 4,000 |
59-
| claude-3.5 Haiku | 7,000 | shared | 4,000 |
60-
| **claude-3.5 Sonnet (New)** | **15,000** | **30,000** | **4,000** |
61-
| Google Gemini 2.0 Flash | 7,000 | shared | 4,000 |
62-
| Google Gemini 2.0 Flash-Lite Preview | 7,000 | shared | 4,000 |
52+
| **Model** | **Conversation Context** | **@-mention Context** | **Output** |
53+
| ----------------------------- | ------------------------ | --------------------- | ---------- |
54+
| GPT 4o mini | 7,000 | shared | 4,000 |
55+
| GPT o3 mini medium | 7,000 | shared | 4,000 |
56+
| GPT 4 Turbo | 7,000 | shared | 4,000 |
57+
| GPT 4o | 7,000 | shared | 4,000 |
58+
| o1 | 7,000 | shared | 4,000 |
59+
| o3 mini high | 7,000 | shared | 4,000 |
60+
| Claude 3.5 Haiku | 7,000 | shared | 4,000 |
61+
| **Claude 3.5 Sonnet (New)** | **15,000** | **30,000** | **4,000** |
62+
| Claude 3.7 Sonnet | 15,000 | 30,000 | 4,000 |
63+
| Gemini 2.0 Flash | 7,000 | shared | 4,000 |
64+
| Gemini 2.0 Flash-Lite Preview | 7,000 | shared | 4,000 |
6365

6466
</Tab>
6567
</Tabs>

‎docs/cody/enterprise/model-configuration.mdx‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -293,6 +293,41 @@ In the example above:
293293
- Sourcegraph-provided models are used for `"chat"` and `"fastChat"` (accessed via Cody Gateway)
294294
- The newly configured model, `"huggingface-codellama::v1::CodeLlama-7b-hf"`, is used for `"autocomplete"` (connecting directly to Hugging Face’s OpenAI-compatible API)
295295

296+
#### Example configuration with Claude 3.7 Sonnet
297+
298+
```json
299+
{
300+
"modelRef": "anthropic::2024-10-22::claude-3-7-sonnet-latest",
301+
"displayName": "Claude 3.7 Sonnet",
302+
"modelName": "claude-3-7-sonnet-latest",
303+
"capabilities": [
304+
"chat",
305+
"reasoning"
306+
],
307+
"category": "accuracy",
308+
"status": "stable",
309+
"tier": "pro",
310+
"contextWindow": {
311+
"maxInputTokens": 45000,
312+
"maxOutputTokens": 4000
313+
},
314+
"modelCost": {
315+
"unit": "mtok",
316+
"inputTokenPennies": 300,
317+
"outputTokenPennies": 1500
318+
},
319+
"reasoningEffort": "high"
320+
},
321+
```
322+
323+
In this modelOverrides config example:
324+
325+
- The model is configured to use Claude 3.7 Sonnet with Cody Gateway
326+
- The model is configured to use the `"chat"` and `"reasoning"` capabilities
327+
- The `reasoningEffort` can be set to 3 different options in the Model Config. These options are `high`, `medium` and `low`
328+
- The default `reasoningEffort` is set to `low`
329+
- When the reasoning effort is `low`, 1024 tokens is used as the thinking budget. With `medium` and `high` the thinking budget is set via `max_tokens_to_sample/2`
330+
296331
Refer to the [examples page](/cody/enterprise/model-config-examples) for additional examples.
297332

298333
## View configuration

‎src/utils/constants/supportedModelsEnt.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ export const chatTableDataEnt: any = {
2626
{ provider: 'Anthropic', model: 'claude-3 Sonnet', status: '✅ *(5.3.9104 and above)' },
2727
{ provider: 'Anthropic', model: 'claude-3.5 Sonnet', status: '✅ *(5.5.0 and above)' },
2828
{ provider: 'Anthropic', model: 'claude-3.5 Sonnet (Latest)', status: '✅ *(5.9 and above)' },
29+
{ provider: 'Anthropic', model: 'claude-3.7 Sonnet', status: '✅ *(6.1.1295 and above)' },
2930
{ provider: 'Google', model: 'Gemini 1.5 Pro', status: '✅ *(5.4.5099 and above)' },
3031
],
3132
'OpenAI': [
@@ -52,6 +53,7 @@ export const chatTableDataEnt: any = {
5253
{ provider: 'Anthropic', model: 'claude-3 Sonnet', status: '✅ *(5.3.9104 and above)' },
5354
{ provider: 'Anthropic', model: 'claude-3.5 Sonnet', status: '✅ *(5.5.0 and above)' },
5455
{ provider: 'Anthropic', model: 'claude-3.5 Sonnet (Latest)', status: '✅ *(5.5.0 and above)' },
56+
{ provider: 'Anthropic', model: 'claude-3.7 Sonnet', status: '✅ *(6.1.1295 and above)' },
5557
{ provider: 'Google', model: 'Gemini 1.5 Pro', status: '❌' },
5658
],
5759
'Google': [

0 commit comments

Comments
 (0)