Skip to content

Commit 94c1aa3

Browse files
authored
Merge pull request #837 from nipype/enh/env-doc
doc: minicline pass at improving environment docs
2 parents b0a9552 + f6d6548 commit 94c1aa3

File tree

1 file changed

+94
-3
lines changed

1 file changed

+94
-3
lines changed

docs/source/explanation/environments.rst

Lines changed: 94 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -13,20 +13,111 @@ to the execution call (e.g. ``my_task(worker="cf", environment="docker")``) or i
1313
Specifying at execution
1414
-----------------------
1515

16-
Work in progress...
16+
The environment for a task can be specified at execution time by passing the ``environment`` argument to the task call.
17+
This can be an instance of `pydra.environments.native.Environment` (for the host system),
18+
`pydra.environments.docker.Environment` (for Docker containers), or
19+
`pydra.environments.singularity.Environment` (for Singularity containers), or a custom environment.
20+
21+
Example:
22+
23+
.. code-block:: python
24+
25+
from pydra.environments import native, docker, singularity
26+
from pydra.compose import shell
27+
# Define a simple shell task
28+
shelly = shell.fuse("echo hello")
29+
30+
# Execute with a native environment
31+
outputs_native = shelly(environment=native.Environment())
32+
33+
# Execute with a Docker environment (assuming busybox image is available)
34+
outputs_docker = shelly(environment=docker.Environment(image="busybox"))
35+
36+
# Execute with a Singularity environment (assuming an image is available)
37+
outputs_singularity = shelly(environment=singularity.Environment(image="/path/to/image.sif"))
38+
39+
Alternatively, when using a `pydra.engine.submitter.Submitter`, the environment can be specified in the Submitter constructor:
40+
41+
.. code-block:: python
42+
43+
from pydra.engine.submitter import Submitter
44+
from pydra.environments import native
45+
from pydra.compose import shell
46+
47+
shelly = shell.fuse("echo hello")
48+
with Submitter(environment=native.Environment()) as sub:
49+
result = sub(shelly)
1750
1851
1952
Specifying at workflow construction
2053
-----------------------------------
2154

22-
Work in progress...
55+
When constructing a workflow, the environment can be specified in the ``workflow.add()`` call.
56+
This ensures that all tasks within that workflow branch will execute in the specified environment.
57+
58+
Example:
59+
60+
.. code-block:: python
61+
62+
from pydra.environments import singularity
63+
from pydra.compose import workflow, shell
64+
from fileformats.generic import File
2365
66+
image = "/path/to/my_singularity_image.sif" # Replace with your Singularity image path
67+
68+
Singu = shell.define("cat {file}")
69+
70+
def MyWorkflow(file: File) -> str:
71+
singu_task = workflow.add(
72+
Singu(file=file),
73+
environment=singularity.Environment(image=image),
74+
)
75+
return singu_task.stdout
76+
77+
# Now you can use MyWorkflow, and the 'cat' task will run in the Singularity environment
2478
2579
2680
Implementing new environment types
2781
----------------------------------
2882

29-
Work in progress...
83+
Custom environment types can be implemented by creating a new class that inherits from `pydra.environments.Environment`.
84+
These custom environment classes are typically located in the `pydra/environments/` directory.
85+
86+
Example (simplified custom environment):
87+
88+
.. code-block:: python
89+
90+
from pydra.environments import Environment as PydraEnvironment
91+
92+
class MyCustomEnvironment(PydraEnvironment):
93+
def __init__(self, some_config: str):
94+
super().__init__()
95+
self.some_config = some_config
96+
97+
def _setup(self):
98+
# Logic to set up the custom environment
99+
print(f"Setting up custom environment with config: {self.some_config}")
100+
101+
def _execute(self, command: list):
102+
# Logic to execute a command within the custom environment
103+
# This is where you would integrate with a custom execution system
104+
print(f"Executing command: {' '.join(command)} in custom environment")
105+
# For demonstration, just return a dummy result
106+
return {"stdout": "Custom environment output", "return_code": 0}
107+
108+
def _tear_down(self):
109+
# Logic to tear down the custom environment
110+
print("Tearing down custom environment")
111+
112+
Then, you can use your custom environment like any other built-in environment:
113+
114+
.. code-block:: python
115+
116+
from pydra.compose import shell
117+
# Assume MyCustomEnvironment is defined as above
118+
my_task = shell.fuse("echo Hello from custom env")
119+
outputs = my_task(environment=MyCustomEnvironment(some_config="test"))
120+
print(outputs.stdout)
30121
31122
32123
.. _Docker: https://www.docker.com/

0 commit comments

Comments
 (0)