Skip to content

Commit 7acb55d

Browse files
committed
update info about number of tasks
1 parent 74a806c commit 7acb55d

File tree

1 file changed

+16
-4
lines changed

1 file changed

+16
-4
lines changed

_episodes/03-configuration.md

Lines changed: 16 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ Let's change our working directory to ESMValTool, then make a copy of the file a
2828
This file contains the information for:
2929
* Rootpath to input data
3030
* Directory structure for the data from different projects
31-
* Number of available CPUs
31+
* Number of parallel tasks
3232
* Destination directory
3333
* Auxiliary data directory
3434
* Output settings
@@ -69,7 +69,7 @@ Add the root path of the folder where you downloaded the data during the [Setup]
6969

7070
> ## Setting the correct rootpath
7171
>
72-
> * For more information about setting the rootpath, you can visit ESMValTool [documentation](https://esmvaltool.readthedocs.io/projects/esmvalcore/en/latest/esmvalcore/datafinder.html).
72+
> For more information about setting the rootpath, you can visit ESMValTool [documentation](https://esmvaltool.readthedocs.io/projects/esmvalcore/en/latest/esmvalcore/datafinder.html).
7373
{: .callout}
7474

7575
## Directory structure for the data from different projects
@@ -83,10 +83,22 @@ drs:
8383

8484
> ## Available drs
8585
>
86-
> * For more information about directories, you can visit ESMValTool [documentation](https://esmvaltool.readthedocs.io/projects/esmvalcore/en/latest/esmvalcore/config.html#developer-configuration-file).
86+
> For more information about directories, you can visit ESMValTool [documentation](https://esmvaltool.readthedocs.io/projects/esmvalcore/en/latest/esmvalcore/config.html#developer-configuration-file).
87+
{: .callout}
88+
89+
## Number of parallel tasks
90+
This option enables you to perform parallel processing. You can choose the number of tasks in parallel as 1/2/3/4/... or you can set it to ``null`` that tells ESMValTool to use the number of available CPUs:
91+
92+
~~~
93+
max_parallel_tasks: null
94+
~~~
95+
{: .source}
96+
97+
> ## Set the number of tasks
98+
>
99+
> If you run out of memory, try setting ``max_parallel_tasks`` to 1. Then, check the amount of memory you need for that by inspecting the file ``run/resource_usage.txt`` in the output directory. Using the number there you can increase the number of parallel tasks again to a reasonable number for the amount of memory available in your system.
87100
{: .callout}
88101

89-
## Number of available CPUs
90102
## Auxiliary data directory (used for some additional datasets)
91103
auxiliary_data_dir: ~/auxiliary_data
92104

0 commit comments

Comments
 (0)