Skip to content

Commit 4dcc14b

Browse files
committed
edited wording + added note
1 parent eac8eac commit 4dcc14b

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

samples/managed-llm/README.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,13 +4,16 @@
44

55
This sample application demonstrates the use of OpenAI-compatible Managed LLMs (Large Language Models) with Defang.
66

7+
8+
> Note: Using Docker Model Provider? See our [*Managed LLM with Docker Model Provider*](https://github.com/DefangLabs/samples/tree/main/samples/managed-llm-provider) sample.
9+
710
The OpenAI-compatible managed LLM feature, provided by the Defang OpenAI Access Gateway, allows users to use AWS Bedrock or Google Cloud Vertex AI with an OpenAI compatible SDK. This enables switching from OpenAI to one of these cloud-native platforms without modifying your application code.
811

912
You can configure the `MODEL` and `ENDPOINT_URL` for the LLM separately for local development and production environments.
1013
* The `MODEL` is the LLM Model ID you are using.
1114
* The `ENDPOINT_URL` is the bridge that provides authenticated access to the LLM model.
1215

13-
Ensure you have the necessary permissions to access the model you intend to use. To do this, you can check your [AWS Bedrock model access](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access-modify.html) or [GCP Vertex AI model access](https://cloud.google.com/vertex-ai/generative-ai/docs/control-model-access).
16+
Ensure you have enabled model access for the model you intend to use. To do this, you can check your [AWS Bedrock model access](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access-modify.html) or [GCP Vertex AI model access](https://cloud.google.com/vertex-ai/generative-ai/docs/control-model-access).
1417

1518
### Defang OpenAI Access Gateway
1619

0 commit comments

Comments
 (0)