Skip to content

Commit bd02c21

Browse files
rsmonterodmingolla
rsmontero
and
dmingolla
committed
F #1550, #6885: New scheduler and monitor documentation
- Update index of components for monitor and scheduler. - Reviewed Overview, Configuration, Rank and OneDRS schedulers - Reviewed Monitoring guides overview - Add new integration guide for schedulers Co-autored-by: Mirko Stojiljkovic <[email protected]> Co-autored-by: Marco Mancini <[email protected]> Co-autored-by: MarioRobres <[email protected]> Co-authored-by: Daniele <[email protected]>
1 parent 976716c commit bd02c21

File tree

33 files changed

+1496
-785
lines changed

33 files changed

+1496
-785
lines changed

‎source/images/scheduler_architecture.png‎

61.5 KB
Loading

‎source/installation_and_configuration/data_center_federation/overview.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ The cloud administrators can share appliances across different Zones with :ref:`
2323

2424
.. important::
2525

26-
Although a single Sunstone server can connect to different Zones, all the other OpenNebula services will only work with the local Zone resources. This includes the :ref:`Scheduler <schg>`, :ref:`OneFlow <oneapps_overview>`, and :ref:`OneGate <onegate_overview>`.
26+
Although a single Sunstone server can connect to different Zones, all the other OpenNebula services will only work with the local Zone resources. This includes the :ref:`Scheduler <scheduler_overview>`, :ref:`OneFlow <oneapps_overview>`, and :ref:`OneGate <onegate_overview>`.
2727

2828
How Should I Read This Chapter
2929
================================================================================

‎source/installation_and_configuration/frontend_installation/install.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -172,7 +172,7 @@ Check how to :ref:`change oneadmin password <change_credentials>` for already ru
172172

173173
.. note::
174174

175-
For advanced setup, follow the configuration references for OpenNebula :ref:`Daemon <oned_conf>` and :ref:`Scheduler <schg>`.
175+
For advanced setup, follow the configuration references for OpenNebula :ref:`Daemon <oned_conf>`.
176176

177177
FireEdge
178178
--------

‎source/installation_and_configuration/opennebula_services/index.rst‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,6 @@ OpenNebula Services
99

1010
Overview <overview>
1111
OpenNebula Configuration <oned>
12-
Scheduler Configuration <scheduler>
13-
Monitoring Configuration <monitoring>
1412
FireEdge Configuration <fireedge>
1513
OneFlow Configuration <oneflow>
1614
OneGate Configuration <onegate>

‎source/installation_and_configuration/opennebula_services/overview.rst‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,6 @@ How Should I Read This Chapter
1212
Read the sections for services you are interested in
1313

1414
- :ref:`OpenNebula Daemon <oned_conf>`
15-
- :ref:`Scheduler <sched_conf>`
1615
- :ref:`Monitoring <mon_conf>`
1716
- :ref:`Sunstone <sunstone_conf>`
1817
- :ref:`FireEdge <fireedge_conf>`

‎source/installation_and_configuration/opennebula_services/troubleshooting.rst‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,6 @@ Configure Logging System
1515
Follow the guides of each component to find the logs' location and configuration of log verbosity:
1616

1717
- OpenNebula Daemon: :ref:`logs <oned_conf_service>`, :ref:`configuration <oned_conf>` (parameter ``LOG/DEBUG_LEVEL``)
18-
- Scheduler: :ref:`logs <sched_conf_service>`, :ref:`configuration <sched_conf>` (parameter ``LOG/DEBUG_LEVEL``)
1918
- Monitoring: :ref:`logs <mon_conf_service>`, :ref:`configuration <mon_conf>` (parameter ``LOG/DEBUG_LEVEL``)
2019
- FireEdge: :ref:`logs <fireedge_conf_service>`, :ref:`configuration <fireedge_conf>` (parameter ``log``)
2120
- OneFlow: :ref:`logs <oneflow_conf_service>`, :ref:`configuration <oneflow_conf>` (parameter ``:debug_level``)

‎source/integration_and_development/infrastructure_drivers_development/devel-sched.rst‎

Lines changed: 348 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,348 @@
1+
.. _devel_sched:
2+
3+
================================================================================
4+
Scheduler Driver
5+
================================================================================
6+
7+
The scheduler driver allows you to develop custom schedulers to work with OpenNebula. A scheduler driver is an executable that generated plans to be executed by ``oned``. A plan is a set of actions to be executed over a VM (e.g. migrate a VM to another host).
8+
9+
10+
Scheduler Driver Actions
11+
================================================================================
12+
13+
The main drivers are located under ``/var/lib/one/remotes/scheduler/<scheduler_driver>``. The default installation of OpenNebula includes the ``dummy``, ``rank`` and ``one_drs`` schedulers. A scheduler needs to implement two actions:
14+
15+
- **OPTIMIZE:** This action optimizes the workload of a cluster. Its outcomes are:
16+
17+
- **SUCCESS:** A plan is returned and attached to the cluster. The plan can be applied either automatically or manually (after user review).
18+
- **FAILURE:** The optimization cannot be computed; an error is returned and attached to the cluster.
19+
20+
**Format:** ``OPTIMIZE <CLUSTER_ID> <SCHED_ACTION_DOCUMENT>``
21+
22+
- **PLACE:** This action allocates VMs in the *PENDING* state and re-schedules VMs flagged for rescheduling.
23+
24+
- **SUCCESS:** A plan is returned and automatically executed. (Note: A successful status may include VMs that could not be allocated if free resources are lacking.)
25+
- **FAILURE:** An error is returned, and the failure is logged to inform the user.
26+
27+
**Format:** ``PLACE - <SCHED_ACTION_DOCUMENT>``
28+
29+
30+
Data Model
31+
--------------------------------------------------------------------------------
32+
33+
The scheduler driver communicates using an XML-based protocol defined in a ``<SCHED_ACTION_MESSAGE>`` which includes the following elements:
34+
35+
- **/VM_POOL/VM:** Lists all VMs matching the scheduling request:
36+
37+
- For PLACE: VMs in *PENDING* or *RESCHED* states.
38+
- For OPTIMIZE: VMs running in the specified cluster.
39+
40+
41+
- **/HOST_POOL/HOST:** Lists hosts to consider:
42+
43+
- For PLACE: all available hosts.
44+
- For OPTIMIZE: only hosts in the target cluster.
45+
46+
47+
- **/DATASTORE_POOL/DATASTORE:** Lists datastores:
48+
49+
- For PLACE: all datastores.
50+
- For OPTIMIZE: only cluster-associated datastores.
51+
52+
53+
- **/VNET_POOL/VNET:** Lists virtual networks:
54+
55+
- For PLACE: all networks.
56+
- For OPTIMIZE: only cluster-associated networks.
57+
58+
59+
- **/VMGROUP_POOL/VMGROUP:** Lists defined VM groups.
60+
61+
62+
- **/REQUIREMENTS/VM/:** Contains placement requirements for each VM, such as:
63+
64+
- ``<ID>``: VM identifier.
65+
- ``<HOSTS>/<ID>``: IDs of eligible hosts.
66+
- ``<NIC>/<ID>``: NIC identifier.
67+
- ``<NIC>/<VNETS>/<ID>``: Virtual network ID for the NIC.
68+
- ``<DATASTORE>/<ID>``: Datastore ID.
69+
70+
71+
- **/CLUSTER:** The cluster document, including:
72+
73+
- ``TEMPLATE/ONE_DRS``: OneDRS configuration (e.g. MIGRATION_THRESHOLD, POLICY, COST_FUNCTION, MODE).
74+
- ``PLAN``: The previous optimization plan (if any), which may be reused for faster re-optimization.
75+
76+
77+
Example of an ``<SCHEDULER_DRIVER_ACTION>``:
78+
79+
.. code-block:: xml
80+
81+
<?xml version="1.0"?>
82+
<SCHEDULER_DRIVER_ACTION>
83+
<VM_POOL>
84+
<VM>
85+
<ID>0</ID>
86+
<UID>0</UID>
87+
<GID>0</GID>
88+
<UNAME>oneadmin</UNAME>
89+
<GNAME>oneadmin</GNAME>
90+
<NAME>testvm-0</NAME>
91+
<PERMISSIONS>
92+
...
93+
</PERMISSIONS>
94+
<LAST_POLL>1743093418</LAST_POLL>
95+
<STATE>3</STATE>
96+
<LCM_STATE>3</LCM_STATE>
97+
<PREV_STATE>3</PREV_STATE>
98+
<PREV_LCM_STATE>3</PREV_LCM_STATE>
99+
<RESCHED>0</RESCHED>
100+
<STIME>1743093368</STIME>
101+
<ETIME>0</ETIME>
102+
<DEPLOY_ID>host0:testvm-0:dummy</DEPLOY_ID>
103+
<MONITORING>
104+
<CPU><![CDATA[0]]></CPU>
105+
...
106+
<TIMESTAMP><![CDATA[1743093418]]></TIMESTAMP>
107+
</MONITORING>
108+
<SCHED_ACTIONS/>
109+
<TEMPLATE>
110+
<AUTOMATIC_REQUIREMENTS><![CDATA[!(PIN_POLICY = PINNED)]]></AUTOMATIC_REQUIREMENTS>
111+
<CPU><![CDATA[0.1]]></CPU>
112+
<MEMORY><![CDATA[128]]></MEMORY>
113+
<VMID><![CDATA[0]]></VMID>
114+
</TEMPLATE>
115+
<USER_TEMPLATE/>
116+
<HISTORY_RECORDS>
117+
...
118+
</HISTORY_RECORDS>
119+
<BACKUPS>
120+
<BACKUP_CONFIG/>
121+
<BACKUP_IDS/>
122+
</BACKUPS>
123+
</VM>
124+
<VM>
125+
...
126+
</VM>
127+
</VM_POOL>
128+
<HOST_POOL>
129+
<HOST>
130+
<ID>0</ID>
131+
<NAME>host0</NAME>
132+
<STATE>2</STATE>
133+
<PREV_STATE>2</PREV_STATE>
134+
<IM_MAD><![CDATA[dummy]]></IM_MAD>
135+
<VM_MAD><![CDATA[dummy]]></VM_MAD>
136+
<CLUSTER_ID>100</CLUSTER_ID>
137+
<CLUSTER>test_cluster</CLUSTER>
138+
<HOST_SHARE>
139+
<MEM_USAGE>1048576</MEM_USAGE>
140+
<CPU_USAGE>80</CPU_USAGE>
141+
<TOTAL_MEM>4005824</TOTAL_MEM>
142+
<TOTAL_CPU>200</TOTAL_CPU>
143+
<MAX_MEM>4005824</MAX_MEM>
144+
<MAX_CPU>200</MAX_CPU>
145+
<RUNNING_VMS>8</RUNNING_VMS>
146+
<VMS_THREAD>1</VMS_THREAD>
147+
<DATASTORES>
148+
<DISK_USAGE><![CDATA[0]]></DISK_USAGE>
149+
<DS>
150+
<FREE_MB><![CDATA[56766]]></FREE_MB>
151+
<ID><![CDATA[0]]></ID>
152+
<TOTAL_MB><![CDATA[63328]]></TOTAL_MB>
153+
<USED_MB><![CDATA[6546]]></USED_MB>
154+
</DS>
155+
<FREE_DISK><![CDATA[56766]]></FREE_DISK>
156+
<MAX_DISK><![CDATA[63328]]></MAX_DISK>
157+
<USED_DISK><![CDATA[6546]]></USED_DISK>
158+
</DATASTORES>
159+
<PCI_DEVICES/>
160+
<NUMA_NODES>
161+
...
162+
</NUMA_NODES>
163+
</HOST_SHARE>
164+
<VMS>
165+
<ID>0</ID>
166+
<ID>1</ID>
167+
<ID>2</ID>
168+
<ID>3</ID>
169+
<ID>4</ID>
170+
<ID>5</ID>
171+
<ID>6</ID>
172+
<ID>7</ID>
173+
</VMS>
174+
<TEMPLATE>
175+
<ARCH><![CDATA[x86_64]]></ARCH>
176+
<CGROUPS_VERSION><![CDATA[2]]></CGROUPS_VERSION>
177+
<CPUSPEED><![CDATA[0]]></CPUSPEED>
178+
<HOSTNAME><![CDATA[ubuntu2204-kvm-ssh-6-99-c94e-1.test]]></HOSTNAME>
179+
...
180+
</TEMPLATE>
181+
<MONITORING>
182+
<TIMESTAMP>1743093419</TIMESTAMP>
183+
<ID>0</ID>
184+
<CAPACITY>
185+
<FREE_CPU><![CDATA[2]]></FREE_CPU>
186+
<FREE_MEMORY><![CDATA[3573260]]></FREE_MEMORY>
187+
<USED_CPU><![CDATA[198]]></USED_CPU>
188+
<USED_MEMORY><![CDATA[432564]]></USED_MEMORY>
189+
</CAPACITY>
190+
<SYSTEM>
191+
<NETRX><![CDATA[4751228]]></NETRX>
192+
<NETTX><![CDATA[9932392]]></NETTX>
193+
</SYSTEM>
194+
<NUMA_NODE>
195+
<HUGEPAGE>
196+
<FREE><![CDATA[0]]></FREE>
197+
<SIZE><![CDATA[2048]]></SIZE>
198+
</HUGEPAGE>
199+
...
200+
<MEMORY>
201+
<FREE><![CDATA[3170204]]></FREE>
202+
<USED><![CDATA[835620]]></USED>
203+
</MEMORY>
204+
<NODE_ID><![CDATA[0]]></NODE_ID>
205+
</NUMA_NODE>
206+
</MONITORING>
207+
<CLUSTER_TEMPLATE>
208+
...
209+
</CLUSTER_TEMPLATE>
210+
</HOST>
211+
<HOST>
212+
...
213+
</HOST>
214+
</HOST_POOL>
215+
<DATASTORE_POOL>
216+
<DATASTORE>
217+
<ID>0</ID>
218+
<UID>0</UID>
219+
<GID>0</GID>
220+
<UNAME>oneadmin</UNAME>
221+
<GNAME>oneadmin</GNAME>
222+
<NAME>system</NAME>
223+
<PERMISSIONS>
224+
...
225+
</PERMISSIONS>
226+
<DS_MAD><![CDATA[-]]></DS_MAD>
227+
<TM_MAD><![CDATA[dummy]]></TM_MAD>
228+
<BASE_PATH><![CDATA[/var/lib/one//datastores/0]]></BASE_PATH>
229+
<TYPE>1</TYPE>
230+
<DISK_TYPE>0</DISK_TYPE>
231+
<STATE>0</STATE>
232+
<CLUSTERS>
233+
<ID>0</ID>
234+
</CLUSTERS>
235+
<TOTAL_MB>4796800</TOTAL_MB>
236+
<FREE_MB>3333260</FREE_MB>
237+
<USED_MB>1429920</USED_MB>
238+
<IMAGES/>
239+
<TEMPLATE>
240+
<ALLOW_ORPHANS><![CDATA[NO]]></ALLOW_ORPHANS>
241+
<DS_MIGRATE><![CDATA[YES]]></DS_MIGRATE>
242+
<SHARED><![CDATA[YES]]></SHARED>
243+
<TM_MAD><![CDATA[dummy]]></TM_MAD>
244+
<TYPE><![CDATA[SYSTEM_DS]]></TYPE>
245+
</TEMPLATE>
246+
</DATASTORE>
247+
<DATASTORE>
248+
...
249+
</DATASTORE>
250+
</DATASTORE_POOL>
251+
<VNET_POOL/>
252+
<VM_GROUP_POOL/>
253+
<CLUSTER_POOL>
254+
<CLUSTER>
255+
<ID>100</ID>
256+
<NAME>test_cluster</NAME>
257+
<HOSTS>
258+
<ID>0</ID>
259+
<ID>1</ID>
260+
<ID>2</ID>
261+
<ID>3</ID>
262+
</HOSTS>
263+
<DATASTORES>
264+
<ID>100</ID>
265+
</DATASTORES>
266+
<VNETS/>
267+
<TEMPLATE>
268+
<ONE_DRS>
269+
<AUTOMATION><![CDATA[full]]></AUTOMATION>
270+
<CPU_USAGE_WEIGHT><![CDATA[0.2]]></CPU_USAGE_WEIGHT>
271+
<CPU_WEIGHT><![CDATA[0.2]]></CPU_WEIGHT>
272+
<DISK_WEIGHT><![CDATA[0.1]]></DISK_WEIGHT>
273+
<MEMORY_WEIGHT><![CDATA[0.4]]></MEMORY_WEIGHT>
274+
<MIGRATION_THRESHOLD><![CDATA[10]]></MIGRATION_THRESHOLD>
275+
<NET_WEIGHT><![CDATA[0.1]]></NET_WEIGHT>
276+
<POLICY><![CDATA[balance]]></POLICY>
277+
<PREDICTIVE><![CDATA[0.2]]></PREDICTIVE>
278+
</ONE_DRS>
279+
<RESERVED_CPU><![CDATA[]]></RESERVED_CPU>
280+
<RESERVED_MEM><![CDATA[]]></RESERVED_MEM>
281+
</TEMPLATE>
282+
</CLUSTER>
283+
</CLUSTER_POOL>
284+
<REQUIREMENTS>
285+
<VM>
286+
<ID>0</ID>
287+
<HOSTS>
288+
<ID>0</ID>
289+
<ID>1</ID>
290+
<ID>2</ID>
291+
<ID>3</ID>
292+
</HOSTS>
293+
<DATASTORES>
294+
<ID>100</ID>
295+
</DATASTORES>
296+
</VM>
297+
<VM>
298+
...
299+
</VM>
300+
</REQUIREMENTS>
301+
</SCHEDULER_DRIVER_ACTION>
302+
303+
304+
The result of a scheduling action is an XML plan document. This plan specifies the operations to be executed on VMs and includes detailed information about each action.
305+
306+
- **PLAN/ID:** Cluster ID which the plan is applied for (``-1`` for initial placement actions)
307+
308+
- **ACTION:** Each Plan action contains:
309+
310+
- ``VM_ID``: Identifier of the target VM.
311+
- ``OPERATION``: The operation to perform (e.g., ``deploy``, ``migrate``, ``poweroff``).
312+
- ``HOST_ID/DS_ID``: For operations like deploy and migrate, the target host and datastore are specified.
313+
- ``NIC``: (For deploy operations) Contains one or more NIC configurations with:
314+
315+
- ``NIC_ID``: Identifier of the NIC.
316+
- ``NETWORK_ID``: The associated virtual network.
317+
318+
Example of an XML Plan:
319+
320+
.. code-block:: xml
321+
322+
<PLAN>
323+
<ID>-1</ID>
324+
<ACTION>
325+
<VM_ID>23</VM_ID>
326+
<OPERATION>deploy</OPERATION>
327+
<HOST_ID>12</HOST_ID>
328+
<DS_ID>100</DS_ID>
329+
<NIC>
330+
<NIC_ID>0</NIC_ID>
331+
<NETWORK_ID>101</NETWORK_ID>
332+
</NIC>
333+
<NIC>
334+
<NIC_ID>1</NIC_ID>
335+
<NETWORK_ID>100</NETWORK_ID>
336+
</NIC>
337+
</ACTION>
338+
<ACTION>
339+
<VM_ID>24</VM_ID>
340+
<OPERATION>migrate</OPERATION>
341+
<HOST_ID>15</HOST_ID>
342+
<DS_ID>200</DS_ID>
343+
</ACTION>
344+
<ACTION>
345+
<VM_ID>25</VM_ID>
346+
<OPERATION>poweroff</OPERATION>
347+
</ACTION>
348+
</PLAN>

‎source/integration_and_development/infrastructure_drivers_development/index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,3 +13,4 @@ Infrastructure Integration
1313
Authentication Driver <devel-auth>
1414
Market Driver <devel-market>
1515
IPAM Driver <devel-ipam>
16+
Scheduler Driver <devel-sched>

0 commit comments

Comments
 (0)