Skip to content

Commit c3bca4a

Browse files
bergerhofferbergerhoffer
committed
OSDOCS-2364: Docs for secondary scheduler operator
1 parent acd0c19 commit c3bca4a

16 files changed

+364
-0
lines changed

‎_attributes/common-attributes.adoc‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,8 @@ endif::[]
3636
:sandboxed-containers-version: 1.1
3737
:sandboxed-containers-legacy-version: 1.0.2
3838
:cert-manager-operator: cert-manager Operator for Red Hat OpenShift
39+
:secondary-scheduler-operator-full: Secondary Scheduler Operator for Red Hat OpenShift
40+
:secondary-scheduler-operator: Secondary Scheduler Operator
3941
:rh-virtualization-first: Red Hat Virtualization (RHV)
4042
:rh-virtualization: RHV
4143
:rh-virtualization-engine-name: Manager

‎_topic_maps/_topic_map.yml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1918,6 +1918,17 @@ Topics:
19181918
File: nodes-custom-scheduler
19191919
- Name: Evicting pods using the descheduler
19201920
File: nodes-descheduler
1921+
- Name: Secondary scheduler
1922+
Dir: secondary_scheduler
1923+
Topics:
1924+
- Name: Secondary scheduler overview
1925+
File: index
1926+
- Name: Secondary Scheduler Operator release notes
1927+
File: nodes-secondary-scheduler-release-notes
1928+
- Name: Scheduling pods using a secondary scheduler
1929+
File: nodes-secondary-scheduler-configuring
1930+
- Name: Uninstalling the Secondary Scheduler Operator
1931+
File: nodes-secondary-scheduler-uninstalling
19211932
- Name: Using Jobs and DaemonSets
19221933
Dir: jobs
19231934
Topics:

‎modules/nodes-secondary-scheduler-about.adoc‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * nodes/scheduling/secondary_scheduler/index.adoc
4+
5+
:_content-type: CONCEPT
6+
[id="nodes-secondary-scheduler-about_{context}"]
7+
= About the {secondary-scheduler-operator}
8+
9+
The {secondary-scheduler-operator-full} provides a way to deploy a custom secondary scheduler in {product-title}. The secondary scheduler runs alongside the default scheduler to schedule pods. Pod configurations can specify which scheduler to use.
10+
11+
The custom scheduler must have the `/bin/kube-scheduler` binary and be based on the link:https://kubernetes.io/docs/concepts/scheduling-eviction/scheduling-framework/[Kubernetes scheduling framework].
12+
13+
[IMPORTANT]
14+
====
15+
You can use the {secondary-scheduler-operator} to deploy a custom secondary scheduler in {product-title}, but Red Hat does not directly support the functionality of the custom secondary scheduler.
16+
====
17+
18+
The {secondary-scheduler-operator} creates the default roles and role bindings required by the secondary scheduler. You can specify which scheduling plug-ins to enable or disable by configuring the `KubeSchedulerConfiguration` resource for the secondary scheduler.

‎modules/nodes-secondary-scheduler-configuring-console.adoc‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * nodes/scheduling/secondary_scheduler/nodes-secondary-scheduler-configuring.adoc
4+
5+
:_content-type: PROCEDURE
6+
[id="nodes-secondary-scheduler-configuring-console_{context}"]
7+
= Deploying a secondary scheduler
8+
9+
After you have installed the {secondary-scheduler-operator}, you can deploy a secondary scheduler.
10+
11+
.Prerequisities
12+
13+
* You have access to the cluster with `cluster-admin` privileges.
14+
* You have access to the {product-title} web console.
15+
* The {secondary-scheduler-operator-full} is installed.
16+
17+
.Procedure
18+
19+
. Log in to the {product-title} web console.
20+
. Create config map to hold the configuration for the secondary scheduler.
21+
.. Navigate to *Workloads* -> *ConfigMaps*.
22+
.. Click *Create ConfigMap*.
23+
.. In the YAML editor, enter the config map definition that contains the necessary `KubeSchedulerConfiguration` configuration. For example:
24+
+
25+
[source,yaml]
26+
----
27+
apiVersion: v1
28+
kind: ConfigMap
29+
metadata:
30+
name: "secondary-scheduler-config" <1>
31+
namespace: "openshift-secondary-scheduler-operator" <2>
32+
data:
33+
"config.yaml": |
34+
apiVersion: kubescheduler.config.k8s.io/v1beta3
35+
kind: KubeSchedulerConfiguration <3>
36+
leaderElection:
37+
leaderElect: false
38+
profiles:
39+
- schedulerName: secondary-scheduler <4>
40+
plugins: <5>
41+
score:
42+
disabled:
43+
- name: NodeResourcesBalancedAllocation
44+
- name: NodeResourcesLeastAllocated
45+
----
46+
<1> The name of the config map. This is used in the *Scheduler Config* field when creating the `SecondaryScheduler` CR.
47+
<2> The config map must be created in the `openshift-secondary-scheduler-operator` namespace.
48+
<3> The `KubeSchedulerConfiguration` resource for the secondary scheduler. For more information, see link:https://kubernetes.io/docs/reference/config-api/kube-scheduler-config.v1beta3/#kubescheduler-config-k8s-io-v1beta3-KubeSchedulerConfiguration[`KubeSchedulerConfiguration`] in the Kubernetes API documentation.
49+
<4> The name of the secondary scheduler. Pods that set their `spec.schedulerName` field to this value are scheduled with this secondary scheduler.
50+
<5> The plug-ins to enable or disable for the secondary scheduler. For a list default scheduling plug-ins, see link:https://kubernetes.io/docs/reference/scheduling/config/#scheduling-plugins[Scheduling plugins] in the Kubernetes documentation.
51+
52+
.. Click *Create*.
53+
54+
. Create the `SecondaryScheduler` CR:
55+
.. Navigate to *Operators* -> *Installed Operators*.
56+
.. Select *{secondary-scheduler-operator-full}*.
57+
.. Select the *Secondary Scheduler* tab and click *Create SecondaryScheduler*.
58+
.. The *Name* field defaults to `cluster`; do not change this name.
59+
.. The *Scheduler Config* field defaults to `secondary-scheduler-config`. Ensure that this value matches the name of the config map created earlier in this procedure.
60+
.. In the *Scheduler Image* field, enter the image name for your custom scheduler.
61+
+
62+
[IMPORTANT]
63+
====
64+
Red Hat does not directly support the functionality of your custom secondary scheduler.
65+
====
66+
67+
.. Click *Create*.

‎modules/nodes-secondary-scheduler-install-console.adoc‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * nodes/scheduling/secondary_scheduler/nodes-secondary-scheduler-configuring.adoc
4+
5+
:_content-type: PROCEDURE
6+
[id="nodes-secondary-scheduler-install-console_{context}"]
7+
= Installing the {secondary-scheduler-operator}
8+
9+
You can use the web console to install the {secondary-scheduler-operator-full}.
10+
11+
.Prerequisites
12+
13+
* You have access to the cluster with `cluster-admin` privileges.
14+
* You have access to the {product-title} web console.
15+
16+
.Procedure
17+
18+
. Log in to the {product-title} web console.
19+
20+
. Create the required namespace for the {secondary-scheduler-operator-full}.
21+
.. Navigate to *Administration* -> *Namespaces* and click *Create Namespace*.
22+
.. Enter `openshift-secondary-scheduler-operator` in the *Name* field and click *Create*.
23+
+
24+
// There are no metrics to collect for the secondary scheduler operator as of now, so no need to add the metrics label
25+
26+
. Install the {secondary-scheduler-operator-full}.
27+
.. Navigate to *Operators* -> *OperatorHub*.
28+
.. Enter *{secondary-scheduler-operator-full}* into the filter box.
29+
.. Select the *{secondary-scheduler-operator-full}* and click *Install*.
30+
.. On the *Install Operator* page:
31+
... The *Update channel* is set to *stable*, which installs the latest stable release of the {secondary-scheduler-operator-full}.
32+
... Select *A specific namespace on the cluster* and select *openshift-secondary-scheduler-operator* from the drop-down menu.
33+
... Select an *Update approval* strategy.
34+
+
35+
* The *Automatic* strategy allows Operator Lifecycle Manager (OLM) to automatically update the Operator when a new version is available.
36+
* The *Manual* strategy requires a user with appropriate credentials to approve the Operator update.
37+
... Click *Install*.
38+
39+
.Verification
40+
41+
. Navigate to *Operators* -> *Installed Operators*.
42+
. Verify that *{secondary-scheduler-operator-full}* is listed with a *Status* of *Succeeded*.

‎modules/nodes-secondary-scheduler-pod-console.adoc‎

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * nodes/scheduling/secondary_scheduler/nodes-secondary-scheduler-configuring.adoc
4+
5+
:_content-type: PROCEDURE
6+
[id="nodes-secondary-scheduler-pod-console_{context}"]
7+
= Scheduling a pod using the secondary scheduler
8+
9+
To schedule a pod using the secondary scheduler, set the `schedulerName` field in the pod definition.
10+
11+
.Prerequisities
12+
13+
* You have access to the cluster with `cluster-admin` privileges.
14+
* You have access to the {product-title} web console.
15+
* The {secondary-scheduler-operator-full} is installed.
16+
* A secondary scheduler is configured.
17+
18+
.Procedure
19+
20+
. Log in to the {product-title} web console.
21+
. Navigate to *Workloads* -> *Pods*.
22+
. Click *Create Pod*.
23+
. In the YAML editor, enter the desired pod configuration and add the `schedulerName` field:
24+
+
25+
[source,yaml]
26+
----
27+
apiVersion: v1
28+
kind: Pod
29+
metadata:
30+
name: nginx
31+
namespace: default
32+
spec:
33+
containers:
34+
- name: nginx
35+
image: nginx:1.14.2
36+
ports:
37+
- containerPort: 80
38+
schedulerName: secondary-scheduler <1>
39+
----
40+
<1> The `schedulerName` field must match the name that is defined in the config map when you configured the secondary scheduler.
41+
42+
. Click *Create*.
43+
44+
.Verification
45+
46+
. Log in to the OpenShift CLI.
47+
. Describe the pod using the following command:
48+
+
49+
[source,terminal]
50+
----
51+
$ oc describe pod nginx -n default
52+
----
53+
+
54+
.Example output
55+
[source,text]
56+
----
57+
Name: nginx
58+
Namespace: default
59+
Priority: 0
60+
Node: ci-ln-t0w4r1k-72292-xkqs4-worker-b-xqkxp/10.0.128.3
61+
...
62+
Events:
63+
Type Reason Age From Message
64+
---- ------ ---- ---- -------
65+
Normal Scheduled 12s secondary-scheduler Successfully assigned default/nginx to ci-ln-t0w4r1k-72292-xkqs4-worker-b-xqkxp
66+
...
67+
----
68+
69+
. In the events table, find the event with a message similar to `Successfully assigned <namespace>/<pod_name> to <node_name>`.
70+
. In the "From" column, verify that the event was generated from the secondary scheduler and not the default scheduler.
71+
+
72+
[NOTE]
73+
====
74+
You can also check the `secondary-scheduler-*` pod logs in the `openshift-secondary-scheduler-namespace` to verify that the pod was scheduled by the secondary scheduler.
75+
====
76+
77+
////
78+
Due to a UI bug, can't verify via console. Bug should be fixed in 4.11 hopefully, and if so, update to use the console steps:
79+
80+
.Verification
81+
. Navigate to the *Events* tab for the pod.
82+
. Find the event with a message similar to `Successfully assigned <namespace>/<pod_name> to <node_name>`.
83+
. Verify that the event was generated from the secondary scheduler and not the default scheduler.
84+
////

‎modules/nodes-secondary-scheduler-remove-resources-console.adoc‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * nodes/scheduling/secondary_scheduler/nodes-secondary-scheduler-uninstalling.adoc
4+
5+
:_content-type: PROCEDURE
6+
[id="nodes-secondary-scheduler-remove-resources-console_{context}"]
7+
= Removing {secondary-scheduler-operator} resources
8+
9+
Optionally, after uninstalling the {secondary-scheduler-operator-full}, you can remove its related resources from your cluster.
10+
11+
.Prerequisites
12+
13+
* You have access to the cluster with `cluster-admin` privileges.
14+
* You have access to the {product-title} web console.
15+
16+
.Procedure
17+
18+
. Log in to the {product-title} web console.
19+
20+
. Remove CRDs that were installed by the {secondary-scheduler-operator}:
21+
.. Navigate to *Administration* -> *CustomResourceDefinitions*.
22+
.. Enter `SecondaryScheduler` in the *Name* field to filter the CRDs.
23+
.. Click the Options menu {kebab} next to the *SecondaryScheduler* CRD and select *Delete Custom Resource Definition*:
24+
25+
. Remove the `openshift-secondary-scheduler-operator` namespace.
26+
.. Navigate to *Administration* -> *Namespaces*.
27+
.. Click the Options menu {kebab} next to the *openshift-secondary-scheduler-operator* and select *Delete Namespace*.
28+
.. In the confirmation dialog, enter `openshift-secondary-scheduler-operator` in the field and click *Delete*.

‎modules/nodes-secondary-scheduler-uninstall-console.adoc‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * nodes/scheduling/secondary_scheduler/nodes-secondary-scheduler-uninstalling.adoc
4+
5+
:_content-type: PROCEDURE
6+
[id="nodes-secondary-scheduler-uninstall-console_{context}"]
7+
= Uninstalling the {secondary-scheduler-operator}
8+
9+
You can uninstall the {secondary-scheduler-operator-full} by using the web console.
10+
11+
.Prerequisites
12+
13+
* You have access to the cluster with `cluster-admin` privileges.
14+
* You have access to the {product-title} web console.
15+
* The {secondary-scheduler-operator-full} is installed.
16+
17+
.Procedure
18+
19+
. Log in to the {product-title} web console.
20+
. Uninstall the {secondary-scheduler-operator-full} Operator.
21+
.. Navigate to *Operators* -> *Installed Operators*.
22+
.. Click the Options menu {kebab} next to the *{secondary-scheduler-operator}* entry and click *Uninstall Operator*.
23+
.. In the confirmation dialog, click *Uninstall*.

‎nodes/scheduling/secondary_scheduler/_attributes‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../../../_attributes

‎nodes/scheduling/secondary_scheduler/images‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../../../images

0 commit comments

Comments
 (0)