You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: modules/developer-lightspeed/proc-changing-your-llm-provider.adoc
+23-8Lines changed: 23 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,20 +16,35 @@ The LLM provider configuration section includes a mandatory dummy provider block
16
16
17
17
.Procedure
18
18
19
-
You can define additional LLM providers using one of two methods.
19
+
You can define additional LLM providers using either of following methods:
20
20
21
-
* Recommended: In your Developer Lightspeed plugin configuration (For example, the `lightspeed` section within the `lightspeed-app-config.yaml` file), define the new provider or providers under the `lightspeed.servers` key as shown in the following code:
21
+
* Recommended: In your Developer Lightspeed plugin configuration (the `lightspeed` section within the `lightspeed-app-config.yaml` file), define the new provider or providers under the `lightspeed.servers` key as shown in the following code:
22
22
+
23
23
[source,yaml]
24
24
----
25
25
lightspeed:
26
26
servers:
27
-
- id: my-new-provider
28
-
url: my-new-url
29
-
token: my-new-token
27
+
- id: _<my_new_provider>_
28
+
url: _<my_new_url>_
29
+
token: _<my_new_token>_
30
+
----
31
+
+
32
+
[NOTE]
33
+
====
34
+
In Developer preview, only one LLM server is supported at a time.
35
+
====
36
+
** Optional: You can set the `id`, `url`, and `token` values in a Kubernetes Secret and reference them as environment variables using the `envFrom` section.
37
+
[source,yaml]
38
+
----
39
+
containers:
40
+
- name: my-container
41
+
image: my-image
42
+
envFrom:
43
+
- secretRef:
44
+
name: my-secret
30
45
----
31
46
32
-
* Alternatively, you can add new LLM providers by updating the rcsconfig.yaml file.
47
+
* You can add new LLM providers by updating the `rcsconfig.yaml` file.
33
48
.. In the `llm_providers` section within your `rcsconfig.yaml` file, add your new provider configuration below the mandatory dummy provider block as shown in the following code:
Copy file name to clipboardExpand all lines: modules/developer-lightspeed/proc-installing-and-configuring-lightspeed.adoc
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -72,7 +72,7 @@ data:
72
72
+
73
73
[IMPORTANT]
74
74
====
75
-
Do not remove the block in the `llm_providers` section. This requirement is crucial when working with {ls-short} because of limitations discovered in Road Core. For instances where you decide to use an alternative LLM provider, you should refer to additional information in the guide. For more information, see link:{developer-lightspeed-link}#proc-changing-your-llm-provider[Changing your LLM provider].
75
+
Do not remove the block in the `llm_providers` section. This requirement is crucial when working with {ls-short} because of limitations discovered in Road Core. For instances where you decide to use an alternative LLM provider, you should refer to additional information in the guide. For more information, see {developer-lightspeed-link}#proc-changing-your-llm-provider_customizing-developer-lightspeed[Changing your LLM provider].
76
76
====
77
77
.. Optional: Configure the number of workers that scale the REST API by specifying the following example to the `ols_config.max_workers` parameter in the {rcs-short} ConfigMap.
0 commit comments