Skip to content

Commit b86de8f

Browse files
BSynRedhatBSynRedhat
authored
Update to LMEval allowOnline setting instructions (#901)
* Update to LMEval allowOnline setting instructions * Updated files * Updates to layout * Further small updates * Updating acc to CodeRabbit suggestions * further tweaks * Tweaks as a result of QE feedback * Updated the instances of Job * further tweaks * addressing codeRabbit comments for newly updated module setting-up-lmeval * UPdated to address peer review comments
1 parent 678c258 commit b86de8f

6 files changed

+215
-44
lines changed

‎.DS_Store

-14 KB
Binary file not shown.

‎assemblies/enabling-external-resource-access-for-lmeval-jobs.adoc

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
:_module-type: ASSEMBLY
2+
3+
ifdef::context[:parent-context: {context}]
4+
[id="enabling-external-resource-access-for-lmeval-jobs_{context}"]
5+
= Enabling external resource access for LMEval jobs
6+
7+
[role='_abstract']
8+
LMEval jobs do not allow internet access or remote code execution by default. When configuring an `LMEvalJob`, it may require access to external resources, for example task datasets and model tokenizers, usually hosted on link:https://huggingface.co[Hugging Face]. If you trust the source and have reviewed the content of these artifacts, an `LMEvalJob` can be configured to automatically download them.
9+
10+
Follow the steps below to enable online access and remote code execution for LMEval jobs. Choose to update these settings by using either the CLI or in the console. Enable one or both settings according to your needs.
11+
12+
include::modules/updating-lmeval-job-configuration-using-the-cli.adoc[leveloffset=+1]
13+
14+
include::modules/updating-lmeval-job-configuration-using-the-web-console.adoc[leveloffset=+1]
15+

‎assemblies/evaluating-large-language-models.adoc

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,8 @@ The following sections show you how to create an `LMEvalJob` custom resource (CR
1717

1818
include::modules/setting-up-lmeval.adoc[leveloffset=+1]
1919

20+
include::./enabling-external-resource-access-for-lmeval-jobs.adoc[leveloffset=+1]
21+
2022
include::modules/lmeval-evaluation-job.adoc[leveloffset=+1]
2123

2224
include::modules/lmeval-evaluation-job-properties.adoc[leveloffset=+1]

‎modules/setting-up-lmeval.adoc

Lines changed: 5 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,6 @@ The following information explains how to create an `LMEvalJob` custom resource
1616

1717
ifdef::upstream[]
1818
[NOTE]
19-
2019
--
2120
LM-Eval is only available in the latest community builds. To use LM-Eval on {productname-long}, ensure that you use ODH 2.20 or later versions and add the following `devFlag` to your `DataScienceCluster` resource:
2221
[source]
@@ -46,7 +45,7 @@ You can configure the following properties for LM-Eval:
4645
4746
| `lmes-detect-device`
4847
| `true/false`
49-
| Detect if there are GPUs available and assign a value for `--device argument` for LM Evaluation Harness. If GPUs are available, the value is `cuda`. If there are no GPUs available, the value is `cpu`.
48+
| Detect if there are GPUs available and assign a value for the `--device argument` for LM Evaluation Harness. If GPUs are available, the value is `cuda`. If there are no GPUs available, the value is `cpu`.
5049
5150
| `lmes-pod-image`
5251
| `quay.io/trustyai/ta-lmes-job:latest`
@@ -73,56 +72,18 @@ You can configure the following properties for LM-Eval:
7372
| The interval to check the job pod for an evaluation job.
7473
7574
| `lmes-allow-online`
76-
| true
75+
| `false`
7776
| Whether LMEval jobs can set the online mode to `on` to access artifacts (models, datasets, tokenizers) from the internet.
7877
79-
| `lmes-code-execution`
80-
| true
78+
| `lmes-allow-code-execution`
79+
| `false`
8180
| Determines whether LMEval jobs can set the `trust remote code` mode to `on`.
8281
8382
|===
8483
8584
After updating the settings in the `ConfigMap`, restart the operator to apply the new values.
8685
87-
88-
// Notes on the 'allow online' setting and patches
89-
90-
// upstream - allowOnline setting is enabled; end-user can disable if they wish.
91-
ifdef::upstream[]
92-
[IMPORTANT]
93-
--
94-
The `allowOnline` setting is enabled by default in {productname-long}. Using `allowOnline` gives the job permissions to automatically download artifacts from external sources. Change this setting to `false` if you do not want LM-Eval to access external sources.
95-
--
96-
endif::[]
97-
98-
// downstream - allowOnline is disabled; end-user needs include a patch in the ConfigMap in order to enable it.
99-
100-
ifndef::upstream[]
10186
[IMPORTANT]
10287
--
103-
The `allowOnline` and `allowCodeExecution` settings are *disabled* by default at the operator level in {productname-long}. This disallows job permissions to automatically download artifacts from external sources.
88+
The `allowOnline` and `allowCodeExecution` settings are *disabled* by default at the operator level in {productname-long}. LMEval jobs do not allow internet access or permission to run any externally downloaded code unless explicitly enabled.
10489
--
105-
106-
.Enabling allowOnline and allowCodeExecution modes
107-
108-
You can set your `LMEvalJob` to access online content by updating and patching the TrustyAI operator `ConfigMap`.
109-
110-
. In the `ConfigMap`, change the `opendatahub.io/managed` setting to `'false'`.
111-
112-
. Patch the TrustyAI operator `ConfigMap` with the following `kubectl` code:
113-
+
114-
[source]
115-
----
116-
kubectl patch configmap trustyai-service-operator-config -n redhat-ods-applications \
117-
--type merge -p '{"data":{"lmes-allow-online":"true","lmes-allow-code-execution":"true"}}'
118-
----
119-
+
120-
. Restart the TrustyAI operator with the following `kubectl` command:
121-
+
122-
[source]
123-
----
124-
kubectl rollout restart deployment trustyai-service-operator-controller-manager -n redhat-ods-applications
125-
----
126-
127-
endif::[]
128-

‎modules/updating-lmeval-job-configuration-using-the-cli.adoc

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
:_module-type: PROCEDURE
2+
3+
ifdef::context[:parent-context: {context}]
4+
5+
[id="updating-lmeval-job-configuration-using-the-cli_{context}"]
6+
= Updating LMEval Job configuration using the CLI
7+
8+
[role='_abstract']
9+
Follow these steps to enable online access (`allowOnline`) or remote code execution (`allowCodeExecution`) or both modes using the CLI for LMEval jobs.
10+
11+
[IMPORTANT]
12+
====
13+
Enabling online access or code execution involves a security risk. Only use these configurations if you trust the source(s).
14+
====
15+
16+
.Prerequisites
17+
18+
* You have cluster administrator privileges for your {productname-long} cluster.
19+
20+
ifndef::upstream[]
21+
* You have downloaded and installed the {productname-short} command-line interface (CLI). See link:https://docs.redhat.com/en/documentation/openshift_container_platform/{ocp-latest-version}/html/cli_tools/openshift-cli-oc#installing-openshift-cli[Installing the OpenShift CLI^].
22+
endif::[]
23+
24+
.Procedure
25+
26+
. Add or set the `opendatahub.io/managed` annotation to `false` to allow user changes to the `trustyai-service-operator-config` ConfigMap:
27+
+
28+
[source,sh]
29+
----
30+
oc patch configmap trustyai-service-operator-config -n redhat-ods-applications \
31+
--type merge -p '{"metadata": {"annotations": {"opendatahub.io/managed": "false"}}}'
32+
----
33+
+
34+
[NOTE]
35+
--
36+
If the `opendatahub.io/managed` annotation does not exist, the default value is `true`.
37+
--
38+
+
39+
.trustyai-service-operator-config annotation keys for LMEval job access
40+
[cols="3,2,5", options="header"]
41+
|===
42+
| Annotation Key
43+
| Value
44+
| Effect
45+
46+
| `opendatahub.io/managed`
47+
| `true`
48+
| User changes are not allowed and will be automatically reverted.
49+
50+
| `opendatahub.io/managed`
51+
| `false`
52+
| Allows the user to manage the TrustyAI operator.
53+
|===
54+
. Update the settings for the TrustyAI operator to enable external connectivity or remote code execution, or both:
55+
+
56+
[source,sh]
57+
----
58+
oc patch configmap trustyai-service-operator-config -n redhat-ods-applications \
59+
--type merge -p '{"data":{"lmes-allow-online":"true","lmes-allow-code-execution":"true"}}'
60+
----
61+
+
62+
.trustyai-service-operator-config configuration keys for LMEval job access
63+
[cols="3,2,5", options="header"]
64+
|===
65+
| ConfigMap Key
66+
| Default
67+
| Description
68+
69+
| `lmes-allow-online`
70+
| `false`
71+
| Enables internet access for LMEval jobs at the cluster level.
72+
73+
| `lmes-allow-code-execution`
74+
| `false`
75+
| Allows execution of code downloaded alongside datasets or tokenizers.
76+
|===
77+
. Restart the TrustyAI operator to apply the configuration changes:
78+
+
79+
[source,sh]
80+
----
81+
oc rollout restart deployment trustyai-service-operator-controller-manager -n redhat-ods-applications
82+
----
83+
. Enable online access or code execution or both on specific `LMEvalJob` resources:
84+
85+
.. For LMEval jobs that already exist, apply the following patch:
86+
+
87+
[source,sh]
88+
----
89+
oc patch lmevaljobs example-lmeval -n <user-or-target-namespace> \
90+
--type merge -p '{"spec":{"allowOnline":true,"allowCodeExecution":true}}'
91+
oc get lmevaljobs example-lmeval -n <user-or-target-namespace> \
92+
-o jsonpath='{.spec.allowOnline}{"\n"}{.spec.allowCodeExecution}{"\n"}'
93+
----
94+
95+
.. For new LMEval jobs, apply a YAML that includes the following fields:
96+
+
97+
[source,yaml]
98+
----
99+
apiVersion: trustyai.opendatahub.io/v1alpha1
100+
kind: LMEvalJob
101+
metadata:
102+
name: example-lmeval
103+
namespace: <your-namespace>
104+
spec:
105+
allowOnline: true
106+
allowCodeExecution: true
107+
108+
----
109+
110+
111+

‎modules/updating-lmeval-job-configuration-using-the-web-console.adoc

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
:_module-type: PROCEDURE
2+
3+
ifdef::context[:parent-context: {context}]
4+
[id="updating-lmeval-job-configuration-using-the-web-console_{context}"]
5+
= Updating LMEval job configuration using the web console
6+
7+
[role='_abstract']
8+
Follow these steps to enable online access (`allowOnline`) and remote code execution (`allowCodeExecution`) modes through the {productname-short} web console for LMEval jobs.
9+
10+
[IMPORTANT]
11+
====
12+
Enabling online access or code execution involves a security risk. Only use these configurations if you trust the source(s).
13+
====
14+
15+
.Prerequisites
16+
17+
* You have cluster administrator privileges for your {productname-long} cluster.
18+
19+
.Procedure
20+
. Navigate to *Workloads → ConfigMaps*.
21+
22+
. Select the *`redhat-ods-applications`* namespace.
23+
24+
. Open the *`trustyai-service-operator-config`* ConfigMap.
25+
26+
. Under *Annotations*, add:
27+
+
28+
[source, yaml]
29+
----
30+
opendatahub.io/managed: "false"
31+
----
32+
. Under *Data*, add or update:
33+
+
34+
[source, yaml]
35+
----
36+
lmes-allow-online: "true"
37+
lmes-allow-code-execution: "true"
38+
----
39+
. Save the ConfigMap.
40+
41+
. Navigate to *Workloads → Deployments*.
42+
43+
. Select the *`redhat-ods-applications`* namespace.
44+
45+
. Locate *`trustyai-service-operator-controller-manager`*.
46+
47+
. Click the *Actions* menu and select *Restart rollout*.
48+
+
49+
[NOTE]
50+
--
51+
Each LMEval job must explicitly be set to allow online access and code execution.
52+
--
53+
. Ensure that the following fields are set to `true` to enable online access and code execution for this job when writing your `LMEvalJob` custom resource:
54+
+
55+
[source,yaml]
56+
----
57+
apiVersion: trustyai.opendatahub.io/v1alpha1
58+
kind: LMEvalJob
59+
metadata:
60+
name: example-lmeval
61+
spec:
62+
allowOnline: true
63+
allowCodeExecution: true
64+
65+
----
66+
67+
68+
.Configuration keys for LMEvalJob custom resource
69+
[cols="3,2,5", options="header"]
70+
|===
71+
| Field
72+
| Default
73+
| Description
74+
75+
| `spec.allowOnline`
76+
| `false`
77+
| Enables this job to access the internet (e.g., to download datasets or tokenizers).
78+
79+
| `spec.allowCodeExecution`
80+
| `false`
81+
| Allows this job to run code included with downloaded resources.
82+
|===

0 commit comments

Comments
 (0)