Skip to content

Commit be3be60

Browse files
committed
review updates
1 parent 7778339 commit be3be60

File tree

4 files changed

+7
-11
lines changed

4 files changed

+7
-11
lines changed

docs/concepts/managed-llms/openai-access-gateway.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ The `x-defang-llm` extension is used to configure the appropriate roles and perm
3737

3838
## Model Mapping
3939

40-
Defang supports model mapping through the openai-access-gateway on AWS and GCP. This takes a model with a Docker naming convention (e.g. ai/lama3.3) and maps it to the closest matching model name on the target platform. If no such match can be found it can fallback onto a known existing model (e.g. ai/mistral). These environment variables are USE_MODEL_MAPPING (default to true) and FALLBACK_MODEL (no default), respectively.
40+
Defang supports model mapping through the [openai-access-gateway](https://github.com/DefangLabs/openai-access-gateway) on AWS and GCP. This takes a model with a Docker naming convention (e.g. ai/lama3.3) and maps it to the closest matching model name on the target platform. If no such match can be found it can fallback onto a known existing model (e.g. ai/mistral). These environment variables are USE_MODEL_MAPPING (default to true) and FALLBACK_MODEL (no default), respectively.
4141

4242
## Current Support
4343

docs/tutorials/deploy-openai-apps-aws-bedrock.mdx

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -110,11 +110,11 @@ Choose the correct `MODEL` depending on which cloud provider you are using.
110110
- For **AWS Bedrock**, use a Bedrock model ID (e.g., `anthropic.claude-3-sonnet-20240229-v1:0`) [See available Bedrock models](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html).
111111
:::
112112

113-
Alternatively, Defang supports model mapping through the openai-access-gateway. This takes a model with a Docker naming convention (e.g. ai/lama3.3) and maps it to
113+
Alternatively, Defang supports model mapping through the openai-access-gateway. This takes a model with a Docker naming convention (e.g. `ai/lama3.3`) and maps it to
114114
the closest equilavent on the target platform. If no such match can be found a fallback can be defined to use a known existing model (e.g. ai/mistral). These environment
115115
variables are USE_MODEL_MAPPING (default to true) and FALLBACK_MODEL (no default), respectively.
116116

117-
117+
118118
:::info
119119
# Complete Example Compose File
120120

@@ -151,13 +151,13 @@ services:
151151
| Variable | AWS Bedrock |
152152
|--------------------|-------------|
153153
| `REGION` | Required|
154-
| `MODEL` | Bedrock model ID / Docker model name |
154+
| `MODEL` | Bedrock model ID or Docker model name, for example `meta.llama3-3-70b-instruct-v1:0` or `ai/lama3.3` |
155155

156156
---
157157

158158
You now have a single app that can:
159159

160-
- Talk to **GCP Vertex AI**
160+
- Talk to **AWS Bedrock**
161161
- Use the same OpenAI-compatible client code
162162
- Easily switch cloud providers by changing a few environment variables
163163
:::

docs/tutorials/deploy-openai-apps-gcp-vertex.mdx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -117,7 +117,7 @@ To do this, you can check your [AWS Bedrock model access](https://docs.aws.amazo
117117

118118
Alternatively, Defang supports model mapping through the openai-access-gateway. This takes a model with a Docker naming convention (e.g. ai/lama3.3) and maps it to
119119
the closest matching one on the target platform. If no such match can be found it can fallback onto a known existing model (e.g. ai/mistral). These environment
120-
variables are USE_MODEL_MAPPING (default to true) and FALLBACK_MODEL (no default), respectively.
120+
variables are `USE_MODEL_MAPPING` (default to true) and `FALLBACK_MODEL` (no default), respectively.
121121

122122

123123
:::info
@@ -158,7 +158,7 @@ services:
158158
|--------------------|---------------|
159159
| `GCP_PROJECT_ID` | Required |
160160
| `REGION` | Required |
161-
| `MODEL` | Vertex model / Docker model name |
161+
| `MODEL` | Vertex model or Docker model name, for example `publishers/meta/models/llama-3.3-70b-instruct-maas` or `ai/llama3.3` |
162162

163163
---
164164

docs/tutorials/deploy-openai-apps.mdx

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,3 @@ Defang currently supports LLM using AWS Bedrock and GCP Vertex AI. Follow the li
99

1010
- [AWS Bedrock](/docs/tutorials/deploy-openai-apps-aws-bedrock/)
1111
- [GCP Vertex AI](/docs/tutorials/deploy-openai-apps-gcp-vertex/).
12-
13-
14-
15-

0 commit comments

Comments
 (0)