Skip to content

Commit 355f96f

Browse files
authored
Manual cherrypick (#1419)
1 parent 60a36d3 commit 355f96f

File tree

2 files changed

+24
-9
lines changed

2 files changed

+24
-9
lines changed

modules/developer-lightspeed/proc-changing-your-llm-provider.adoc

Lines changed: 23 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -16,20 +16,35 @@ The LLM provider configuration section includes a mandatory dummy provider block
1616

1717
.Procedure
1818

19-
You can define additional LLM providers using one of two methods.
19+
You can define additional LLM providers using either of following methods:
2020

21-
* Recommended: In your Developer Lightspeed plugin configuration (For example, the `lightspeed` section within the `lightspeed-app-config.yaml` file), define the new provider or providers under the `lightspeed.servers` key as shown in the following code:
21+
* Recommended: In your Developer Lightspeed plugin configuration (the `lightspeed` section within the `lightspeed-app-config.yaml` file), define the new provider or providers under the `lightspeed.servers` key as shown in the following code:
2222
+
2323
[source,yaml]
2424
----
2525
lightspeed:
2626
servers:
27-
- id: my-new-provider
28-
url: my-new-url
29-
token: my-new-token
27+
- id: _<my_new_provider>_
28+
url: _<my_new_url>_
29+
token: _<my_new_token>_
30+
----
31+
+
32+
[NOTE]
33+
====
34+
In Developer preview, only one LLM server is supported at a time.
35+
====
36+
** Optional: You can set the `id`, `url`, and `token` values in a Kubernetes Secret and reference them as environment variables using the `envFrom` section.
37+
[source,yaml]
38+
----
39+
containers:
40+
- name: my-container
41+
image: my-image
42+
envFrom:
43+
- secretRef:
44+
name: my-secret
3045
----
3146

32-
* Alternatively, you can add new LLM providers by updating the rcsconfig.yaml file.
47+
* You can add new LLM providers by updating the `rcsconfig.yaml` file.
3348
.. In the `llm_providers` section within your `rcsconfig.yaml` file, add your new provider configuration below the mandatory dummy provider block as shown in the following code:
3449
+
3550
[source,yaml]
@@ -42,9 +57,9 @@ llm_providers:
4257
models:
4358
- name: dummymodel
4459
# END: Do not remove this block
45-
- name: my-new-providers
60+
- name: _<my_new_providers>_
4661
type: openai
47-
url: my-provider-url
62+
url: _<my_provider_url>_
4863
credentials_path: path/to/token
4964
disable_model_check: true
5065
----

modules/developer-lightspeed/proc-installing-and-configuring-lightspeed.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,7 @@ data:
7272
+
7373
[IMPORTANT]
7474
====
75-
Do not remove the block in the `llm_providers` section. This requirement is crucial when working with {ls-short} because of limitations discovered in Road Core. For instances where you decide to use an alternative LLM provider, you should refer to additional information in the guide. For more information, see link:{developer-lightspeed-link}#proc-changing-your-llm-provider[Changing your LLM provider].
75+
Do not remove the block in the `llm_providers` section. This requirement is crucial when working with {ls-short} because of limitations discovered in Road Core. For instances where you decide to use an alternative LLM provider, you should refer to additional information in the guide. For more information, see {developer-lightspeed-link}#proc-changing-your-llm-provider_customizing-developer-lightspeed[Changing your LLM provider].
7676
====
7777
.. Optional: Configure the number of workers that scale the REST API by specifying the following example to the `ols_config.max_workers` parameter in the {rcs-short} ConfigMap.
7878
+

0 commit comments

Comments
 (0)