You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/cody/enterprise/model-config-examples.mdx
+6-6Lines changed: 6 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,9 +2,9 @@
2
2
3
3
<pclassName="subtitle">This section includes examples about how to configure Cody to use Sourcegraph-provided models with `modelConfiguration`. These examples will use the following:</p>
## Sourcegraph-provided models and BYOK (Bring Your Own Key)
10
10
@@ -69,7 +69,7 @@ In the configuration above, we:
69
69
Below are configuration examples for setting up various LLM providers using BYOK. These examples are applicable whether or not you are using Sourcegraph-supported models.
70
70
71
71
- In this section, all configuration examples have Sourcegraph-provided models disabled. Please refer to the previous section to use a combination of Sourcegraph-provided models and BYOK.
72
-
- Ensure that at least one model is available for each Cody feature ("chat" and "autocomplete"), regardless of the provider and model overrides configured. To verify this, [view the configuration](/cody/model-configuration#view-configuration) and confirm that appropriate models are listed in the `"defaultModels"` section.
72
+
- Ensure that at least one model is available for each Cody feature ("chat" and "autocomplete"), regardless of the provider and model overrides configured. To verify this, [view the configuration](/cody/enterprise/model-configuration#view-configuration) and confirm that appropriate models are listed in the `"defaultModels"` section.
| `endpoint` | For pay-as-you-go, set it to an AWS region code (e.g., `us-west-2`) when using a public Amazon Bedrock endpoint. For provisioned throughput, set it to the provisioned VPC endpoint for the bedrock-runtime API (e.g., `https://vpce-0a10b2345cd67e89f-abc0defg.bedrock-runtime.us-west-2.vpce.amazonaws.com`). |
587
587
| `region` | The region to use when configuring API clients. This is necessary because the 'frontend' binary container cannot access environment variables from the host OS. |
588
588
589
-
Provisioned throughput for AWS Bedrock models can be configured using the `"awsBedrockProvisionedThroughput"` server-side configuration type. Refer to the [Model Overrides](/cody/model-configuration#model-overrides) section for more details.
589
+
Provisioned throughput for AWS Bedrock models can be configured using the `"awsBedrockProvisionedThroughput"` server-side configuration type. Refer to the [Model Overrides](/cody/enterprise/model-configuration#model-overrides) section for more details.
590
590
591
591
</Accordion>
592
592
593
593
## Self-hosted models
594
594
595
595
<Callout type="note">Configuring self-hosted models can be nuanced and complex. Please reach out to Sourcegraph support for assistance with the setup.</Callout>
596
596
597
-
Please refer to the [self-hosted models](/cody/model-configuration#self-hosted-models) documentation for detailed information about configuring self-hosted models.
597
+
Please refer to the [self-hosted models](/cody/enterprise/model-configuration#self-hosted-models) documentation for detailed information about configuring self-hosted models.
|[`sourcegraph`](/cody/model-configuration#sourcegraph-supplied-models)| Configures access to Sourcegraph-provided models available through Cody Gateway. |
20
-
|[`providerOverrides`](/cody/model-configuration#provider-overrides)| Configures model providers, allowing you to customize Cody's connection to model providers using your own API keys or self-hosted models. |
21
-
|[`modelOverrides`](/cody/model-configuration#model-overrides)| Extends or modifies the list of models Cody recognizes and their configurations. |
22
-
|[`selfHostedModels`](/cody/model-configuration#self-hosted-models)| Adds models to Cody’s recognized models list with default configurations provided by Sourcegraph. Only available for certain models; general models can be configured in `modelOverrides`. |
23
-
|[`defaultModels`](/cody/model-configuration#default-models)| Specifies the models assigned to each Cody feature (chat, fast chat, autocomplete). |
19
+
|[`sourcegraph`](/cody/enterprise/model-configuration#sourcegraph-provided-models)| Configures access to Sourcegraph-provided models available through Cody Gateway. |
20
+
|[`providerOverrides`](/cody/enterprise/model-configuration#provider-overrides)| Configures model providers, allowing you to customize Cody's connection to model providers using your own API keys or self-hosted models. |
21
+
|[`modelOverrides`](/cody/enterprise/model-configuration#model-overrides)| Extends or modifies the list of models Cody recognizes and their configurations. |
22
+
|[`selfHostedModels`](/cody/enterprise/model-configuration#self-hosted-models)| Adds models to Cody’s recognized models list with default configurations provided by Sourcegraph. Only available for certain models; general models can be configured in `modelOverrides`. |
23
+
|[`defaultModels`](/cody/enterprise/model-configuration#default-models)| Specifies the models assigned to each Cody feature (chat, fast chat, autocomplete). |
24
24
25
25
## Getting started with `modelConfiguration`
26
26
27
27
The recommended way to set up model configuration is using Sourcegraph-provided models through the Cody Gateway.
28
28
29
-
For a minimal configuration example, see [Configure Sourcegraph-supplied models](/cody/model-configuration#configure-sourcegraph-supplied-models).
29
+
For a minimal configuration example, see [Configure Sourcegraph-supplied models](/cody/enterprise/model-configuration#configure-sourcegraph-provided-models).
30
30
31
31
## Sourcegraph-provided models
32
32
@@ -79,7 +79,7 @@ There are three main settings for configuring Sourcegraph-provided LLM models:
|`endpoint`| (Optional) The URL for connecting to Cody Gateway. The default is set to the production instance. |
81
81
|`accessToken`| (Optional) The access token for connecting to Cody Gateway defaults to the current license key. |
82
-
|[`modelFilters`](/cody/model-configuration#model-filters)| (Optional) Filters specifying which models to include from Cody Gateway. |
82
+
|[`modelFilters`](/cody/enterprise/model-configuration#model-filters)| (Optional) Filters specifying which models to include from Cody Gateway. |
83
83
84
84
### Model Filters
85
85
@@ -166,7 +166,7 @@ In the example above:
166
166
- Sourcegraph-provided models are enabled (`sourcegraph` is not set to `null`)
167
167
- The `"openai"` provider configuration is overridden to be accessed using your own key, with the provider API accessed directly via the specified `endpoint` URL. In contrast, models from the `"anthropic"` provider are accessed through the Cody Gateway
168
168
169
-
Refer to the [examples page](/cody/model-configuration/examples) for additional examples.
169
+
Refer to the [examples page](/cody/enterprise/model-config-examples) for additional examples.
170
170
171
171
### Server-side config
172
172
@@ -176,8 +176,8 @@ Sourcegraph natively supports several types of LLM API providers. The current se
@@ -188,7 +188,7 @@ Sourcegraph natively supports several types of LLM API providers. The current se
188
188
189
189
The configuration for `serverSideConfig` varies by provider type.
190
190
191
-
Refer to the [examples page](/cody/model-configuration/examples) for examples.
191
+
Refer to the [examples page](/cody/enterprise/model-config-examples) for examples.
192
192
193
193
## Model Overrides
194
194
@@ -284,7 +284,7 @@ In the example above:
284
284
- Sourcegraph-provided models are used for `"chat"` and `"fastChat"` (accessed via Cody Gateway)
285
285
- The newly configured model, `"huggingface-codellama::v1::CodeLlama-7b-hf"`, is used for `"autocomplete"` (connecting directly to Hugging Face’s OpenAI-compatible API)
286
286
287
-
Refer to the [examples page](/cody/model-configuration/examples) for additional examples.
287
+
Refer to the [examples page](/cody/enterprise/model-config-examples) for additional examples.
288
288
289
289
## Self-hosted models
290
290
@@ -305,7 +305,7 @@ This field is an array of items, each with the following fields:
305
305
|`apiVersion`| (Optional) The API version to use. Defaults to `"v1"` if not specified. |
306
306
|`model`| (Optional) Specifies which default model configuration to use. Sourcegraph provides default configurations for select models. For arbitrary models, use the `modelOverrides` section. Available options include: `"starcoder2-7b@v1"`, `"starcoder2-15b@v1"`, `"mistral-7b-instruct@v1"`, `"mixtral-8x7b-instruct@v1"`, `"mixtral-8x22b-instruct@v1"`|
307
307
308
-
Refer to the [examples page](/cody/model-configuration/examples#self-hosted-models) for examples.
308
+
Refer to the [examples page](/cody/enterprise/model-config-examples#self-hosted-models) for examples.
0 commit comments