Skip to content

Commit 47bff56

Browse files
author
Alan Christie
committed
docs: Improved README
1 parent 62b7683 commit 47bff56

File tree

1 file changed

+42
-23
lines changed

1 file changed

+42
-23
lines changed

README.rst

Lines changed: 42 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -37,14 +37,14 @@ different name and have more than one.
3737
place to start. The repository is host to a number of job-based
3838
container images and several *manifests* and *job definition files*.
3939

40-
Here's an example **manifest** from the **Virtual Screening** repository::
40+
Here's an example **manifest** from a recent **Virtual Screening** repository::
4141

4242
---
4343
kind: DataManagerManifest
4444
kind-version: '2021.1'
4545

4646
job-definition-files:
47-
- virtual-screening.yaml
47+
- im-virtual-screening.yaml
4848
- rdkit.yaml
4949
- xchem.yaml
5050

@@ -216,21 +216,22 @@ We then place a test in that group with a ``run-group`` declaration
216216
in the corresponding test block::
217217

218218
jobs:
219-
job-a:
219+
max-min-picker:
220220
[...]
221221
tests:
222-
test-1:
222+
test-a:
223223
run-groups:
224224
- name: experiment-a
225225
ordinal: 1
226226

227227
We need to provide an ``ordinal`` value. This numeric value (from 1 ..N)
228228
puts the test in a specific position in the test sequence. When tests are
229-
placed in a ``run-group`` you have to order your tests so that ``a`` follows
230-
``b``. This is done with unique ordinals for each test in each group. A test
231-
with ordinal ``1`` will run before a test with ordinal ``2``.
229+
placed in a ``run-group`` you have to order your tests, i.e. declare that
230+
``test-a`` follows ``test-b``. This is done with unique ordinals for each
231+
test in the ``run-group``. A test with ordinal ``1`` will run before a test
232+
with ordinal ``2``. Ordinals have to be unique within a ``run-group``.
232233

233-
You can run just the tests for a specific group by using the ``--run-group``
234+
You can run the tests for a specific group by using the ``--run-group``
234235
option::
235236

236237
jote --run-group experiment-a
@@ -248,7 +249,7 @@ declaration.
248249
Here we declare a docker-compose file called
249250
``docker-compose-experiment-a.yaml``::
250251

251-
test-groups:
252+
test-groups:
252253
- name: experiment-a
253254
compose:
254255
file: docker-compose-experiment-a.yaml
@@ -265,7 +266,7 @@ you can minimise the risk that your containers are not ready for the tests
265266
by adding a fixed delay between ``jote`` starting the compose file and
266267
running the first test::
267268

268-
test-groups:
269+
test-groups:
269270
- name: experiment-a
270271
compose:
271272
file: docker-compose-experiment-a.yaml
@@ -281,23 +282,41 @@ using the shell, relying on Docker as the execution run-time for the processes
281282
in your workflow.
282283

283284
Be aware that nextflow tests run by ``jote`` run under different conditions
284-
compared to when it runs under the Data Manager's control, where nextflow
285-
will be executed within a Kubernetes environment rather than Docker. This
286-
introduces variability. Nextflow tests that run under ``jote`` *are not*
287-
executed in the same environment or under the same memory or processor
288-
constraints.
289-
290-
When running nextflow jobs ``jote`` writes a ``nextflow.config`` to the
291-
test's simulated project directory prior to executing the command, and
292-
this is the current-working directory when the test starts.
285+
compared to when it runs under the Data Manager's control. In the Data Manager
286+
nextflow jobs will be executed within a Kubernetes environment. When run by ``jote``
287+
nextflow is expected using the operating system shell. This introduces a
288+
variability that you need to take into account - i.e. under ``jote`` the
289+
nextflow controller runs in the shell, and *are not* executed in the same
290+
environment or under the same memory or processor constraints.
291+
292+
You might need to provide a custom nextflow configuration file
293+
for your tests to run successfully. You do this by adding a ``nextflow-config-file``
294+
declaration in the test. Here, we name the file ``nextflow-test.config``::
295+
296+
jobs:
297+
max-min-picker:
298+
[...]
299+
tests:
300+
simple-load:
301+
nextflow-config-file: nextflow-test.config
302+
[...]
303+
304+
The config file must be located in the Job repository's ``data-manager``
305+
directory.
306+
307+
Prior to running the corresponding test ``jote`` copies it to the
308+
Job's project directory as the file ``nextflow.config`` (a standard file
309+
expected by nextflow).
310+
293311
``jote`` *will not* let you have a nextflow config in your home directory
294312
as any settings found there would be merged with the file ``jote`` writes,
295313
potentially disturbing the execution behaviour.
296314

297-
It's your responsibility to install a suitable nextflow that's available
298-
for shell execution when you test any nextflow-type Jobs. ``jote`` expects
299-
to be able to run nextflow when executing the corresponding ``command``
300-
that's defined in the job definition.
315+
.. note::
316+
It's your responsibility to install a suitable nextflow that's available
317+
for shell execution. ``jote`` expects to be able to run nextflow when
318+
executing the corresponding ``command`` that's defined in the job
319+
definition.
301320

302321
Installation
303322
============

0 commit comments

Comments
 (0)