Add modelConfig docs

docs/cody/model-configuration/examples.mdx

@@ -0,0 +1,372 @@
+# Examples
+
+## Sourcegraph-supplied models only
+
+This section includes examples how to configure Cody to use Sourcegraph-supplied models.
+
+-   [Minimal configuration](/cody/model-configuration#configure-sourcegraph-supplied-models)
+-   [Using model filters](/cody/model-configuration#model-filters)
+-   [Change default models](/cody/model-configuration#default-models)
+
+## Sourcegraph-supplied models and BYOK (Bring Your Own Key)
+
+Sourcegraph-supplied models come with preconfigured providers, identified by the following IDs (namespaces):
+
+-   "anthropic"
+-   "google"
+-   "fireworks"
+-   "mistral"
+-   "openai"
+
+### Override provider config for all models in the namespace
+
+When Sourcegraph-supplied models are used and a provider override for a Sourcegraph-supported provider (same ID) is specified,
+the override applies to all Sourcegraph-supplied models within that provider.
+For example, if you specify an override for a provider with ID `"anthropic"`, it will apply to all models from the `"anthropic"` provider.
+
+Example configuration:
+
+```json
+{
+"cody.enabled": true,
+"modelConfiguration": {
+  "sourcegraph": {},
+  "providerOverrides": [
+    {
+     "id": "anthropic",
+    "displayName": "Anthropic BYOK",
+    "serverSideConfig": {
+      "type": "anthropic",
+      "accessToken": "sk-ant-token",
+       "endpoint": "https://api.anthropic.com/v1/messages"
+      }
+    }
+  ],
+  "defaultModels": {
+    "chat": "anthropic::2024-10-22::claude-3.5-sonnet",
+    "fastChat": "anthropic::2023-06-01::claude-3-haiku",
+    "autocomplete": "fireworks::v1::deepseek-coder-v2-lite-base"
+  }
+}
+```
+
+In the configuration above, we:
+
+-   Enable Sourcegraph-supplied models and do not set any overrides (note that `"modelConfiguration.modelOverrides"` is not specified).
+-   Route requests for Anthropic models directly to the Anthropic API (via the provider override specified for "anthropic").
+-   Route requests for other models (such as the Fireworks model for "autocomplete") through Cody Gateway.
+
+### Override provider config for some models in the namespace and use the Sourcegraph-configured provider config for the rest
+
+It's possible to route requests directly to the LLM provider (bypassing the Cody Gateway) for some models while using the
+Sourcegraph-configured provider config for the rest.
+
+Example configuration:
+
+In the configuration above, we:
+
+-   Enable Sourcegraph-supplied models (the `sourcegraph` field is not empty or `null`).
+-   Define a new provider with the ID `"anthropic-byok"` and configure it to use the Anthropic API.
+-   Since this provider is unknown to Sourcegraph, no Sourcegraph-supplied models are available for it.
+    Therefore, we add a custom model in the `"modelOverrides"` section.
+-   Use the custom model configured in the previous step (`"anthropic-byok::2024-10-22::claude-3.5-sonnet"`) for `"chat"`.
+    Requests are sent directly to the Anthropic API as set in the provider override.
+-   For `"fastChat"` and `"autocomplete"`, we use Sourcegraph-supplied models via Cody Gateway.
+
+## Config examples for various LLM providers
+
+Below are configuration examples for setting up various LLM providers using BYOK.
+These examples are applicable whether or not you are using Sourcegraph-supported models.
+
+**Note:**
+
+-   In this section, all configuration examples have Sourcegraph-supplied models disabled. To use a combination of
+    Sourcegraph-supplied models and BYOK, please refer to the previous section.
+-   Ensure that at least one model is available for each Cody feature ("chat", "edit", "autocomplete"), regardless of
+    the provider and model overrides configured. To verify this, [view the configuration](/cody/model-configuration#view-configuration)
+    and confirm that appropriate models are listed in the `"defaultModels"` section.
+
+<Accordion title="Anthropic">
+
+```json
+{
+"cody.enabled": true,
+"modelConfiguration": {
+  "sourcegraph": null,
+  "providerOverrides": [
+    {
+     "id": "anthropic",
+      "displayName": "Anthropic",
+      "serverSideConfig": {
+      "type": "anthropic",
+      "accessToken": "sk-ant-token",
+        "endpoint": "https://api.anthropic.com/v1/messages"
+      }
+    }
+  ],
+  "modelOverrides": [
+    {
+      "modelRef": "anthropic::2024-10-22::claude-3.5-sonnet",
+      "displayName": "Claude 3.5 Sonnet",
+      "modelName": "claude-3-5-sonnet-latest",
+      "capabilities": ["edit", "chat"],
+      "category": "accuracy",
+      "status": "stable",
+      "tier": "free",
+      "contextWindow": {
+        "maxInputTokens": 45000,
+        "maxOutputTokens": 4000
+      }
+    },
+    {
+      "modelRef": "anthropic::2023-06-01::claude-3-haiku",
+      "displayName": "Claude 3 Haiku",
+      "modelName": "claude-3-haiku-20240307",
+      "capabilities": ["edit", "chat"],
+      "category": "speed",
+      "status": "stable",
+      "tier": "free",
+      "contextWindow": {
+          "maxInputTokens": 7000,
+          "maxOutputTokens": 4000
+      }
+    },
+    {
+      "modelRef": "anthropic::2023-01-01::claude-instant-1.2",
+      "displayName": "Claude Instant",
+      "modelName": "claude-instant-1.2",
+      "capabilities": ["autocomplete", "edit", "chat"],
+      "category": "other",
+      "status": "deprecated",
+      "tier": "free",
+      "contextWindow": {
+        "maxInputTokens": 7000,
+        "maxOutputTokens": 4000
+      }
+    }
+  ],
+  "defaultModels": {
+    "chat": "anthropic::2024-10-22::claude-3.5-sonnet",
+    "fastChat": "anthropic::2023-06-01::claude-3-haiku",
+    "autocomplete": "anthropic::2023-01-01::claude-instant-1.2"
+  }
+}
+```
+
+In the configuration above, we:
+
+    -   Set up a provider override for Anthropic, routing requests for this provider directly to the specified Anthropic endpoint (bypassing Cody Gateway).
+    -   Add three Anthropic models:
+        -   Two models with chat capabilities (`"anthropic::2024-10-22::claude-3.5-sonnet"` and `"anthropic::2023-06-01::claude-3-haiku"`),
+            providing options for chat users.
+        -   One model with autocomplete capability (`"anthropic::2023-01-01::claude-instant-1.2"`).
+    -   Set the configured models as default models for Cody features in the `"defaultModels"` field.
+
+</Accordion>
+
+<Accordion title="Fireworks">
+```json
+"cody.enabled": true,
+"modelConfiguration": {
+  "sourcegraph": null,
+  "providerOverrides": [
+    {
+      "id": "fireworks",
+      "displayName": "Fireworks",
+      "serverSideConfig": {
+        "type": "fireworks",
+        "accessToken": "token",
+        "endpoint": "https://api.fireworks.ai/inference/v1/completions"
+      }
+    }
+  ],
+  "modelOverrides": [
+    {
+      "modelRef": "fireworks::v1::mixtral-8x22b-instruct",
+      "displayName": "Mixtral 8x22B",
+      "modelName": "accounts/fireworks/models/mixtral-8x22b-instruct",
+      "capabilities": ["edit", "chat"],
+      "category": "other",
+      "status": "stable",
+      "tier": "free",
+      "contextWindow": {
+          "maxInputTokens": 7000,
+          "maxOutputTokens": 4000
+      }
+    },
+    {
+      "modelRef": "fireworks::v1::starcoder-16b",
+      "modelName": "accounts/fireworks/models/starcoder-16b",
+      "displayName": "(Fireworks) Starcoder 16B",
+      "contextWindow": {
+      "maxInputTokens": 8192,
+      "maxOutputTokens": 4096
+      },
+      "capabilities": ["autocomplete"],
+      "category": "balanced",
+      "status": "stable"
+    }
+  ],
+  "defaultModels": {
+    "chat": "fireworks::v1::mixtral-8x22b-instruct",
+    "fastChat": "fireworks::v1::mixtral-8x22b-instruct",
+    "autocomplete": "fireworks::v1::starcoder-16b"
+  }
+}
+```
+
+In the configuration above, we:
+
+-   Set up a provider override for Fireworks, routing requests for this provider directly to the specified Fireworks endpoint (bypassing Cody Gateway).
+-   Add two Fireworks models: - `"fireworks::v1::mixtral-8x22b-instruct"` with "edit" and "chat" capabiities - used for "chat"
+    and "fastChat" - `"fireworks::v1::starcoder-16b"` with "autocomplete" capability - used for "autocomplete".
+
+</Accordion>
+
+<Accordion title="OpenAI">
+
+```json
+"modelConfiguration": {
+  "sourcegraph": null,
+  "providerOverrides": [
+    {
+      "id": "openai",
+      "displayName": "OpenAI",
+      "serverSideConfig": {
+        "type": "openai",
+        "accessToken": "token",
+        "endpoint": "https://api.openai.com"
+      }
+    }
+  ],
+  "modelOverrides": [
+    {
+      "modelRef": "openai::2024-02-01::gpt-4o",
+      "displayName": "GPT-4o",
+      "modelName": "gpt-4o",
+      "capabilities": ["edit", "chat"],
+      "category": "accuracy",
+      "status": "stable",
+      "tier": "pro",
+      "contextWindow": {
+          "maxInputTokens": 45000,
+          "maxOutputTokens": 400
+      }
+    },
+    {
+      "modelRef": "openai::unknown::gpt-3.5-turbo-instruct",
+      "displayName": "GPT-3.5 Turbo Instruct",
+      "modelName": "gpt-3.5-turbo-instruct",
+      "capabilities": ["autocomplete"],
+      "category": "speed",
+      "status": "stable",
+      "tier": "free",
+      "contextWindow": {
+          "maxInputTokens": 7000,
+          "maxOutputTokens": 400
+      }
+    }
+],
+  "defaultModels": {
+    "chat": "openai::2024-02-01::gpt-4o",
+    "fastChat": "openai::2024-02-01::gpt-4o",
+    "autocomplete": "openai::unknown::gpt-3.5-turbo-instruct"
+  }
+}
+```
+
+In the configuration above, we:
+
+-   Set up a provider override for OpenAI, routing requests for this provider directly to the specified OpenAI endpoint (bypassing Cody Gateway).
+-   Add two OpenAI models:
+    -   `"openai::2024-02-01::gpt-4o"` with "edit" and "chat" capabiities - used for "chat" and "fastChat"
+    -   `"openai::unknown::gpt-3.5-turbo-instruct"` with "autocomplete" capability - used for "autocomplete".
+
+</Accordion>
+
+<Accordion title="Azure OpenAI">
+
+```json
+"cody.enabled": true,
+"modelConfiguration": {
+  "sourcegraph": null,
+  "providerOverrides": [
+      {
+          "id": "azure-openai",
+          "displayName": "Azure OpenAI",
+          "serverSideConfig": {
+              "type": "azureOpenAI",
+              "accessToken": "token",
+              "endpoint": "https://acme-test.openai.azure.com/",
+              "user": "",
+              "useDeprecatedCompletionsAPI": true
+          }
+      }
+  ],
+  "modelOverrides": [
+      {
+          "modelRef": "azure-openai::unknown::gpt-4o",
+          "displayName": "GPT-4o",
+          "modelName": "gpt-4o",
+          "capabilities": ["edit", "chat"],
+          "category": "accuracy",
+          "status": "stable",
+          "tier": "pro",
+          "contextWindow": {
+              "maxInputTokens": 45000,
+              "maxOutputTokens": 4000
+          }
+      },
+          {
+          "modelRef": "azure-openai::unknown::gpt-35-turbo-instruct-test",
+          "displayName": "GPT-3.5 Turbo Instruct",
+          "modelName": "gpt-35-turbo-instruct-test",
+          "capabilities": ["autocomplete"],
+          "category": "speed",
+          "status": "stable",
+          "tier": "free",
+          "contextWindow": {
+              "maxInputTokens": 7000,
+              "maxOutputTokens": 4000
+          }
+      }
+  ],
+  "defaultModels": {
+  "chat": "azure-openai::unknown::gpt-4o",
+  "fastChat": "azure-openai::unknown::gpt-4o",
+  "autocomplete": "azure-openai::unknown::gpt-35-turbo-instruct-test"
+  }
+}
+```
+
+In the configuration above, we:
+
+-   Set up a provider override for Azure OpenAI, routing requests for this provider directly to the specified Azure OpenAI endpoint (bypassing Cody Gateway).
+-   Add two OpenAI models:
+    -   `"azure-openai::unknown::gpt-4o"` with "edit" and "chat" capabiities - used for "chat" and "fastChat"
+    -   `"azure-openai::unknown::gpt-35-turbo-instruct-test"` with "autocomplete" capability - used for "autocomplete".
+-   Since `"azure-openai::unknown::gpt-35-turbo-instruct-test"` is not supported on the newer OpenAI `"v1/chat/completions"` endpoint,
+    we set `"useDeprecatedCompletionsAPI"` to `true` to route requests to the legacy `"v1/completions"` endpoint. This setting is
+    unnecessary if you are using a model supported on the `"v1/chat/completions"` endpoint.
+
+</Accordion>
+
+<Accordion title="Generic OpenAI-compatible">
+
+TODO
+
+</Accordion>
+
+<Accordion title="Google Vertex (Anthropic)"></Accordion>
+<Accordion title="Google Vertex (Gemini)"></Accordion>
+<Accordion title="Google Vertex (public)"></Accordion>
+
+<Accordion title="AWS Bedrock"></Accordion>
+
+## Self-hosted models
+
+TODO
+
+```
+
+```