You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Cyclecloud maintains two resources to expand the dynamic provisioning capability. These resources are *nodearray* and *machinetype*.
43
43
44
-
If you submit a job and specify a nodearray resource by `qsub -l nodearray=highmem -- /bin/hostname`
45
-
then CycleCloud will add nodes to the nodearray named 'highmem'. If there is no such nodearray then the job will remain idle.
44
+
If you submit a job and specify a nodearray resource by `qsub -l nodearray=highmem -- /bin/hostname` then CycleCloud will add nodes to the nodearray named 'highmem'. If there is no such nodearray then the job will remain idle.
46
45
47
46
Similarly if a machinetype resource is specified which a job submission, e.g. `qsub -l machinetype:Standard_L32s_v2 my-job.sh`, then CycleCloud autoscales the 'Standard_L32s_v2' in the 'execute' (default) nodearray. If that machine type is not available in the 'execute' node array then the job will remain idle.
48
47
@@ -51,7 +50,6 @@ These resources can be used in combination as:
which will autoscale only if the 'Standard_HB60rs' machines are specified an the 'hpc' node array.
56
54
57
55
## Adding additional queues assigned to nodearrays
@@ -100,11 +98,7 @@ The following are the PBS Professional specific configuration options you can to
100
98
101
99
## Connect PBS with CycleCloud
102
100
103
-
CycleCloud manages [OpenPBS](http://openpbs.org/) clusters through an installable agent called
104
-
[`azpbs`](https://github.com/Azure/cyclecloud-pbspro). This agent connect to
105
-
CycleCloud to read cluster and VM configurations and also integrates with OpenPBS
106
-
to effectively process the job and host information. All `azpbs` configurations
107
-
are found in the `autoscale.json` file, normally `/opt/cycle/pbspro/autoscale.json`.
101
+
CycleCloud manages [OpenPBS](http://openpbs.org/) clusters through an installable agent called [`azpbs`](https://github.com/Azure/cyclecloud-pbspro). This agent connect to CycleCloud to read cluster and VM configurations and also integrates with OpenPBS to effectively process the job and host information. All `azpbs` configurations are found in the `autoscale.json` file, normally `/opt/cycle/pbspro/autoscale.json`.
108
102
109
103
```
110
104
"password": "260D39rWX13X",
@@ -118,9 +112,7 @@ are found in the `autoscale.json` file, normally `/opt/cycle/pbspro/autoscale.js
118
112
119
113
### Important Files
120
114
121
-
The `azpbs` agent parses the PBS configuration each time it's called - jobs, queues, resources.
122
-
Information is provided in the stderr and stdout of the command as well as to a log file, both
123
-
at configurable levels. All PBS management commands (`qcmd`) with arguments are logged to file as well.
115
+
The `azpbs` agent parses the PBS configuration each time it's called - jobs, queues, resources. Information is provided in the stderr and stdout of the command as well as to a log file, both at configurable levels. All PBS management commands (`qcmd`) with arguments are logged to file as well.
124
116
125
117
All these files can be found in the _/opt/cycle/pbspro/_ directory where the agent is installed.
126
118
@@ -134,10 +126,7 @@ All these files can be found in the _/opt/cycle/pbspro/_ directory where the age
134
126
135
127
136
128
### Defining OpenPBS Resources
137
-
This project allows for a generally association of OpenPBS resources with Azure
138
-
VM resources via the cyclecloud-pbspro (azpbs) project. This resource relationship
139
-
defined in `autoscale.json`.
140
-
129
+
This project allows for a generally association of OpenPBS resources with Azure VM resources via the cyclecloud-pbspro (azpbs) project. This resource relationship defined in `autoscale.json`.
141
130
The default resources defined with the cluster template we ship with are
142
131
143
132
```json
@@ -175,13 +164,9 @@ The default resources defined with the cluster template we ship with are
175
164
}
176
165
```
177
166
178
-
The OpenPBS resource named `mem` is equated to a node attribute named `node.memory`,
179
-
which is the total memory of any virtual machine. This configuration allows `azpbs`
180
-
to process a resource request such as `-l mem=4gb` by comparing the value of the
181
-
job resource requirements to node resources.
167
+
The OpenPBS resource named `mem` is equated to a node attribute named `node.memory`, which is the total memory of any virtual machine. This configuration allows `azpbs` to process a resource request such as `-l mem=4gb` by comparing the value of the job resource requirements to node resources.
182
168
183
-
Note that disk is currently hardcoded to `size::20g`.
184
-
Here is an example of handling VM Size specific disk size
169
+
Note that disk is currently hardcoded to `size::20g`. Here is an example of handling VM Size specific disk size
185
170
```json
186
171
{
187
172
"select": {"node.vm_size": "Standard_F2"},
@@ -197,21 +182,14 @@ Here is an example of handling VM Size specific disk size
197
182
198
183
### Autoscale and Scalesets
199
184
200
-
CycleCloud treats spanning and serial jobs differently in OpenPBS clusters.
201
-
Spanning jobs will land on nodes that are part of the same placement group. The
202
-
placement group has a particular platform meaning (VirtualMachineScaleSet with
203
-
SinglePlacementGroup=true) and CC will managed a named placement group for each
204
-
spanned node set. Use the PBS resource `group_id` for this placement group name.
185
+
CycleCloud treats spanning and serial jobs differently in OpenPBS clusters. Spanning jobs will land on nodes that are part of the same placement group. The placement group has a particular platform meaning VirtualMachineScaleSet with SinglePlacementGroup=true) and CC will managed a named placement group for each spanned node set. Use the PBS resource `group_id` for this placement group name.
205
186
206
-
The `hpc` queue appends
207
-
the equivalent of `-l place=scatter:group=group_id` by using native queue defaults.
187
+
The `hpc` queue appends the equivalent of `-l place=scatter:group=group_id` by using native queue defaults.
208
188
209
189
210
190
### Installing the CycleCloud OpenPBS Agent `azpbs`
211
191
212
-
The OpenPBS CycleCloud cluster will manage the installation and configuration of
213
-
the agent on the server node. The preparation includes setting PBS resources,
214
-
queues, and hooks. A scripted install can be done outside of CycleCloud as well.
192
+
The OpenPBS CycleCloud cluster manages the installation and configuration of the agent on the server node. The preparation includes setting PBS resources, queues, and hooks. A scripted install can be done outside of CycleCloud as well.
215
193
216
194
```bash
217
195
# Prerequisite: python3, 3.6 or newer, must be installed and in the PATH
@@ -246,4 +224,4 @@ azpbs validate
246
224
> CycleCloud does not support the bursting configuration with Open PBS.
247
225
248
226
> [!NOTE]
249
-
> Even though Windows is an officially supported Open PBS platform, CycleCloud does not support running Open PBS on Windows at this time.
227
+
> Even though Windows is an officially supported Open PBS platform, CycleCloud does not support running Open PBS on Windows at this time.
0 commit comments