You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: QuickStart.md
+5-5Lines changed: 5 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,23 +1,23 @@
1
1
# Quick Start Guide
2
2
3
-
This document will guide you through the install procedure and your first hello world example.
3
+
This document will guide you through the install procedure and your first Hello World example.
4
4
5
5
-[Requirements](#requirements)
6
-
-[Install psij](#install-psij)
6
+
-[Install PSI/J](#install-psij)
7
7
-[Hello World example](#hello-world)
8
8
9
9
## Requirements
10
10
- python3.7+
11
11
12
-
## Install psij
12
+
## Install PSI/J
13
13
14
-
If you have conda installed you might want to start from a fresh environment. This part is not installing psij but setting up a new environment with the specified python version:
14
+
If you have conda installed you might want to start from a fresh environment. This part is not installing PSI/J but setting up a new environment with the specified python version:
15
15
16
16
1.`conda create -n psij python=3.7`
17
17
2.`conda activate psij`
18
18
19
19
20
-
Install psij from the GitHub repository:
20
+
Install PSI/J from the GitHub repository:
21
21
22
22
1. Clone the repository into your working directory:
PSI/J provides a common interface for obtaining allocations on compute resources.
14
+
PSI/J provides a common interface for obtaining allocations on compute resources. Usually, those compute resources will already have a batch scheduler in place (for example, SLURM).
15
15
16
-
Usually, those compute resources will already have some batch scheduler in place (for example, SLURM).
17
-
18
-
A PSI/J executor is the code that tells the core of PSI/J how to interact with
19
-
such a batch scheduler so that it can provide a common interface to applications.
16
+
A PSI/J executor is the code that tells the core of PSI/J how to interact with a batch scheduler so that it can provide a common interface to applications.
20
17
21
18
A PSI/J executor needs to implement the abstract methods defined on the :class:`psij.job_executor.JobExecutor` base class.
22
19
The documentation for that class has reference material for each of the methods that won't be repeated here.
23
20
24
21
For batch scheduler systems, the :class:`.BatchSchedulerExecutor` subclass provides further useful structure to help implement JobExecutor.
25
22
This tutorial will focus on using BatchSchedulerExecutor as a base, rather than implementing JobExecutor directly.
26
23
27
-
The batch scheduler executor is based around a model where interactions with a local resource manager happen via command line invocations.
24
+
The batch scheduler executor is based on a model where interactions with a local resource manager happen via command line invocations.
28
25
For example, with PBS `qsub` and `qstat` commands are used to submit a request and to see status.
29
26
30
27
To use BatchSchedulerExecutor for a new local resource manager that uses this command line interface, subclass BatchSchedulerExecutor and add in code that understands how to form the command lines necessary to submit a request for an allocation and to get allocation status. This tutorial will do that for PBSPro.
31
28
29
+
Adding an Executor
30
+
------------------
31
+
32
32
First set up a directory structure::
33
33
34
34
mkdir project/
@@ -44,14 +44,13 @@ We're going to create three source files in this directory structure:
44
44
45
45
* ``psij-descriptors/pbspro_descriptor.py`` - This file tells the PSI/J core what this package implements.
46
46
47
-
First, we'll build a skeleton that won't work, and see that it doesn't work in the test suite. Then we'll build up to the full functionality.
48
-
49
47
Prerequisites:
50
48
51
-
* You have the psij-python package installed already and are able to run whatever basic verification you think is necessary.
49
+
* You have the psij-python package installed and are able to run whatever basic verification you think is necessary.
52
50
53
51
* You are able to submit to PBS Pro on a local system.
54
52
53
+
First, we'll build a skeleton that won't work, and see that it doesn't work in the test suite. Then we'll build up to the full functionality.
55
54
56
55
A Not-implemented Stub
57
56
----------------------
@@ -135,8 +134,7 @@ Now running the same pytest command will give a different error further along in
135
134
136
135
This default BatchSchedulerExecutor code needs a configuration object and none was supplied.
137
136
138
-
A configuration object can contain configuration specific to this particular executor. However,
139
-
for now we are not going to specify a custom configuration object and instead will re-use
137
+
A configuration object can contain configuration specific to this particular executor. For now we are not going to specify a custom configuration object and instead will re-use
140
138
the BatchSchedulerExecutorConfig supplied by the PSI/J core.
141
139
142
140
Define a new __init__ method that will define a default configuration::
@@ -172,15 +170,13 @@ To implement submission, we need to implement three methods:
172
170
173
171
You can read the docstrings for each of these methods for more information, but briefly the submission process is:
174
172
175
-
1. ``generate_submit_script`` should generate a submit script specific to the batch scheduler.
173
+
1. ``generate_submit_script`` generates a submit script specific to the batch scheduler.
176
174
177
-
2. ``get_submit_command`` should return the command line necessary to submit that script to the batch scheduler.
175
+
2. ``get_submit_command`` returns the command line necessary to submit that script to the batch scheduler.
178
176
179
177
The output of that command should be interpreted by ``job_id_from_submit_output`` to extract a batch scheduler specific job ID,
180
178
which can be used later when cancelling a job or getting job status.
181
179
182
-
So let's implement those.
183
-
184
180
In line with other PSI/J executors, we're going to delegate script generation to a template based helper. So add a line to initialize a :py:class:`.TemplatedScriptGenerator` in the
185
181
executor initializer, pointing at a (as yet non-existent) template file, and replace ``generate_submit_script`` with a delegated call to `TemplatedScriptGenerator`::
186
182
@@ -278,7 +274,7 @@ Implementing Status
278
274
279
275
PSI/J needs to ask the batch scheduler for the status of jobs that it has submitted. This can be done with ``BatchSchedulerExecutor`` by overriding these two methods, which we stubbed out as not-implemented earlier on:
280
276
281
-
* :py:meth:`.BatchSchedulerExecutor.get_status_command` - Like ``get_submit_command``, this should return a batch schedulerspecific command line, this time to output job status.
277
+
* :py:meth:`.BatchSchedulerExecutor.get_status_command` - Like ``get_submit_command``, this should return a batch scheduler-specific command line, this time to output job status.
282
278
283
279
* :py:meth:`.BatchSchedulerExecutor.parse_status_output` - This will interpret the output of the above status command, a bit like ``job_id_from_submit_output``.
284
280
@@ -407,7 +403,7 @@ The _STATE_MAP given here is also not exhaustive: if PBS Pro qstat returns a dif
407
403
How to Distribute Your Executor
408
404
-------------------------------
409
405
410
-
If you want to share your executor with others, here are two ways:
406
+
If you want to share your executor with others:
411
407
412
408
1. You can make a Python package and distribute that as an add-on without needing to interact with the PSI/J project.
0 commit comments