Skip to content

Commit 6acd2ee

Browse files
stevsmitstevsmit
authored
Merge pull request #92380 from rh-tokeefe/LCORE-38
LCORE-38: document BYOK
2 parents 1e904f3 + 090cb5d commit 6acd2ee

File tree

3 files changed

+141
-1
lines changed

3 files changed

+141
-1
lines changed

‎configure/ols-configuring-openshift-lightspeed.adoc

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,4 +25,6 @@ include::modules/ols-granting-access-to-individual-users.adoc[leveloffset=+2]
2525
include::modules/ols-granting-access-to-user-group.adoc[leveloffset=+2]
2626
include::modules/ols-filtering-and-redacting-information.adoc[leveloffset=+1]
2727
include::modules/ols-cluster-interaction-overview.adoc[leveloffset=+1]
28-
include::modules/ols-enabling-cluster-interaction.adoc[leveloffset=+2]
28+
include::modules/ols-enabling-cluster-interaction.adoc[leveloffset=+2]
29+
include::modules/ols-about-the-byo-knowledge-tool.adoc[leveloffset=+1]
30+
include::modules/ols-providing-custom-knowledge-to-the-llm.adoc[leveloffset=+2]

‎modules/ols-about-the-byo-knowledge-tool.adoc

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
// Module included in the following assemblies:
2+
// * lightspeed-docs-main/configure/ols-configuring-openshift-lightspeed.adoc
3+
4+
:_mod-docs-content-type: CONCEPT
5+
[id="about-the-byo-knowledge-tool_{context}"]
6+
= About the BYO Knowledge tool
7+
8+
The {ols-long} service uses a large language model (LLM) to generate responses to questions. You can enhance the knowledge that is available to the LLM by using the BYO Knowledge tool to create a retrieval-augmented generation (RAG) database.
9+
10+
When you create a RAG database, you customize the {ols-long} service for your environment. For example, a network administrator can develop a standard operating procedure (SOP) that is used to provision an {ocp-product-title} cluster. Then, the network administrator can use the BYO Knowledge tool to enhance the knowledge available to the LLM by including information from the SOP.
11+
12+
To bring your own knowledge to an LLM, you complete the following steps:
13+
14+
* Create the custom content in Markdown format.
15+
* Use the BYO Knowledge tool to package the content as a container image.
16+
* Push the container image to an image registry, such as `quay.io`.
17+
* Update the `OLSConfig` custom resource file to list the image that you pushed to the image registry.
18+
* Access the {ols-long} virtual assistant and submit a question that is associated with the custom knowledge that you made available to the LLM.
19+
+
20+
[NOTE]
21+
====
22+
When you use the BYO Knowledge tool, the documents that you make available to the LLM are sent to the LLM provider.
23+
====

‎modules/ols-providing-custom-knowledge-to-the-llm.adoc

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
// Module included in the following assemblies:
2+
// * lightspeed-docs-main/configure/ols-configuring-openshift-lightspeed.adoc
3+
4+
:_mod-docs-content-type: PROCEDURE
5+
[id="providing-custom-knowledge-to-the-llm_{context}"]
6+
= Providing custom knowledge to the LLM
7+
8+
Customize the information available to the large language model (LLM) by providing access to a container image that resides in a remote image registry. The examples in this procedure use `quay.io` as the remote container image registry, and the path for the custom image is `quay.io/<username>/my-byok-image:latest`.
9+
10+
:FeatureName: The BYO Knowledge tool
11+
include::snippets/technology-preview.adoc[]
12+
13+
.Prerequisites
14+
15+
* You are logged in to the {ocp-product-title} web console as a user account that has permission to create a cluster-scoped custom resource (CR) file, such as a user with the `cluster-admin` role.
16+
17+
* You have installed the {ols-long} Operator.
18+
19+
* You have configured the LLM provider.
20+
21+
* The custom information you want to add resides as a collection of Markdown files. No other file format is supported.
22+
23+
* You have an account for a container image registry, such as `quay.io`.
24+
25+
.Procedure
26+
27+
. Specify the location of the directory with the Markdown files for the retrieval-augmented generation (RAG) database and the path for the image that the BYO Knowledge tool generates by running the following command:
28+
+
29+
[source,terminal]
30+
----
31+
$ podman run -it --rm --device=/dev/fuse \
32+
-v $XDG_RUNTIME_DIR/containers/auth.json:/run/user/0/containers/auth.json:Z \
33+
-v <dir_tree_with_markdown_files>:/markdown:Z \
34+
-v <dir_for_image_tar>:/output:Z \
35+
registry.redhat.io/openshift-lightspeed-tech-preview/lightspeed-rag-tool-rhel9:latest
36+
----
37+
+
38+
[NOTE]
39+
====
40+
The container image that the tool generates contains a custom RAG database. The RAG database provides additional information to the LLM.
41+
====
42+
43+
. Load the container image that the BYO Knowledge tool generated by running the following command:
44+
+
45+
[source,terminal]
46+
----
47+
$ podman load < <directory_for_image_tar>/<my-byok-image.tar>
48+
----
49+
50+
. Display the podman images that are on your local computer by running the following command:
51+
+
52+
[source,terminal]
53+
----
54+
$ podman images
55+
----
56+
+
57+
.Example output
58+
[source,terminal]
59+
----
60+
REPOSITORY TAG IMAGE ID CREATED SIZE
61+
localhost/my-byok-image latest be7d1770bf10 1 minute ago 2.37 GB
62+
...
63+
----
64+
65+
. Tag the local image with a name and destination so that the image can be pushed to the container image registry by running the following command:
66+
+
67+
[source,terminal]
68+
----
69+
$ podman tag localhost/my-byok-image:latest quay.io/<username>/my-byok-image:latest
70+
----
71+
72+
. Push the local container image to the container image registry by running the following command:
73+
+
74+
[source,terminal]
75+
----
76+
$ podman push quay.io/<username>/my-byok-image:latest
77+
----
78+
79+
. Modify the `OLSconfig` CR to deploy the newly created RAG database alongside the existing one:
80+
81+
.. In the {ocp-product-title} web console, click *Operators* -> *Installed Operators*.
82+
83+
.. Select *All Projects* in the *Project* dropdown at the top of the screen.
84+
85+
.. Click *{ols-long} Operator*.
86+
87+
.. Click *OLSConfig*, then click the `cluster` configuration instance in the list.
88+
89+
.. Click the *YAML* tab.
90+
91+
.. Insert the `spec.ols.rag` yaml code:
92+
+
93+
.Example `OLSconfig` CR file
94+
[source,yaml,subs="attributes,verbatim"]
95+
----
96+
apiVersion: ols.openshift.io/v1alpha1
97+
kind: OLSConfig
98+
metadata:
99+
name: cluster
100+
spec:
101+
ols:
102+
rag:
103+
- image: quay.io/<username>/my-byok-image:latest # <1>
104+
indexPath: /rag/vector_db
105+
indexID: vector_db_index
106+
----
107+
<1> Where `image` specifies the tag for the image that was pushed to the image registry so that the {ols-long} Operator can access the custom content. The {ols-long} Operator can work with more than one RAG database that you create.
108+
109+
. Click *Save*.
110+
111+
.Verification
112+
113+
* Access the {ols-long} virtual assistant and submit a question associated with the custom content that you provided to the LLM.
114+
+
115+
The {ols-long} virtual assistant generates a response based on the custom content.

0 commit comments

Comments
 (0)