You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Pydra is a lightweight Python dataflow engine for scientific analysis.
7
+
Although designed as a successor to Nipype_, Pydra is supports analytics in any domain.
8
+
Pydra helps build reproducible, scalable, reusable workflows that link processing tasks
9
+
implemented in Python or shell commands to be executed on distributed compute platforms.
11
10
12
-
The power of Pydra lies in ease of constructing workflows, containing complex
13
-
multiparameter map-reduce operations, in Python code and the use of a global cache (see
14
-
:ref:`Design philosophy` for the rationale behind its design).
11
+
The power of Pydra lies in ease of constructing workflows containing complex
12
+
multiparameter map-reduce operations in Python code (see :ref:`Design philosophy` for
13
+
the rationale behind its design).
15
14
16
-
**Key features**:
15
+
**Key features:**
17
16
18
17
* Combine diverse tasks (`Python functions <./tutorial/3-python.html>`__ or `shell commands <./tutorial/4-shell.html>`__) into coherent `workflows <./tutorial/5-workflow.html>`__
19
-
* Map-reduce like semantics (see :ref:`Splitting and combining`)
18
+
* Concurrent execution on `choice of computing platform (e.g. workstation, SLURM, SGE, Dask, etc...) <./tutorial/2-advanced-execution.html#Workers>`__
20
19
* Dynamic workflow construction using Python code (see :ref:`Dynamic construction`)
21
-
* Modular backends for deployment on different execution platforms (e.g. cloud, HPC, etc...) (see `Execution options <./tutorial/2-advanced-execution.html>`__)
22
-
* Support for the execution of tasks in containerized environments (see :ref:`Software environments`)
20
+
* Map-reduce-like semantics (see :ref:`Splitting and combining`)
23
21
* Global caching to reduce recomputation (see :ref:`Caches and hashes`)
24
-
* Support for strong type-checking, including file types, at workflow construction time (see :ref:`Typing and file-formats`)
22
+
* Tasks can be executed in separate software environments, e.g. containers (see :ref:`Software environments`)
23
+
* Strong type-checking, including file types, before execution (see :ref:`Typing and file-formats`)
25
24
26
25
27
26
Installation
@@ -32,21 +31,20 @@ therefore, it is straightforward to install via pip for Python >= 3.11
32
31
33
32
.. code-block:: bash
34
33
35
-
$ pip install pydra
34
+
$ pip install pydra
36
35
37
36
Pre-designed tasks are available under the `pydra.tasks.*` package namespace. These tasks
38
-
are implemented within separate packages that are typically specific to a given shell-command toolkit such as FSL_, AFNI_ or ANTs_,
39
-
or a collection of related tasks/workflows (e.g. `niworkflows`_). Pip can be used to
40
-
install these packages as well:
41
-
37
+
are implemented within separate packages that are typically specific to a given
38
+
shell-command toolkit such as FSL_, AFNI_ or ANTs_, or a collection of related
39
+
tasks/workflows (e.g. `niworkflows`_). Pip can be used to install these packages as well:
42
40
43
41
.. code-block:: bash
44
42
45
-
$ pip install pydra-fsl pydra-ants
43
+
$ pip install pydra-fsl pydra-ants
46
44
47
-
Of course, if you use Pydra to execute commands within toolkits, you will need to
48
-
either have those commands installed on the execution machine, or use containers
49
-
environments (see `Environments <../explanation/environments.html>`__) to run them.
45
+
Of course, if you use Pydra to execute commands within non-Python toolkits, you will
46
+
need to either have those commands installed on the execution machine, or use containers
47
+
to run them (see :ref:`Software environments`).
50
48
51
49
52
50
Tutorials and notebooks
@@ -55,10 +53,14 @@ Tutorials and notebooks
55
53
The following tutorials provide a step-by-step guide to using Pydra.
56
54
They can be read in any order, but it is recommended to start with :ref:`Getting started`.
57
55
The tutorials are implemented as Jupyter notebooks, which can be downloaded and run locally
58
-
or run online using the |Binder| within each tutorial.
56
+
or run online using the |Binder| button within each tutorial.
59
57
60
58
If you decide to download the notebooks and run locally, be sure to install the necessary
61
-
dependencies with ``pip install -e /path/to/your/pydra[tutorial]``.
59
+
dependencies with
60
+
61
+
.. code-block:: bash
62
+
63
+
$ pip install -e /path/to/your/pydra[tutorial]
62
64
63
65
64
66
Execution
@@ -67,7 +69,7 @@ Execution
67
69
Learn how to execute existing tasks (including workflows) on different systems
0 commit comments