@@ -13,20 +13,111 @@ to the execution call (e.g. ``my_task(worker="cf", environment="docker")``) or i
13
13
Specifying at execution
14
14
-----------------------
15
15
16
- Work in progress...
16
+ The environment for a task can be specified at execution time by passing the ``environment `` argument to the task call.
17
+ This can be an instance of `pydra.environments.native.Environment ` (for the host system),
18
+ `pydra.environments.docker.Environment ` (for Docker containers), or
19
+ `pydra.environments.singularity.Environment ` (for Singularity containers), or a custom environment.
20
+
21
+ Example:
22
+
23
+ .. code-block :: python
24
+
25
+ from pydra.environments import native, docker, singularity
26
+ from pydra.compose import shell
27
+ # Define a simple shell task
28
+ shelly = shell.fuse(" echo hello" )
29
+
30
+ # Execute with a native environment
31
+ outputs_native = shelly(environment = native.Environment())
32
+
33
+ # Execute with a Docker environment (assuming busybox image is available)
34
+ outputs_docker = shelly(environment = docker.Environment(image = " busybox" ))
35
+
36
+ # Execute with a Singularity environment (assuming an image is available)
37
+ outputs_singularity = shelly(environment = singularity.Environment(image = " /path/to/image.sif" ))
38
+
39
+ Alternatively, when using a `pydra.engine.submitter.Submitter `, the environment can be specified in the Submitter constructor:
40
+
41
+ .. code-block :: python
42
+
43
+ from pydra.engine.submitter import Submitter
44
+ from pydra.environments import native
45
+ from pydra.compose import shell
46
+
47
+ shelly = shell.fuse(" echo hello" )
48
+ with Submitter(environment = native.Environment()) as sub:
49
+ result = sub(shelly)
17
50
18
51
19
52
Specifying at workflow construction
20
53
-----------------------------------
21
54
22
- Work in progress...
55
+ When constructing a workflow, the environment can be specified in the ``workflow.add() `` call.
56
+ This ensures that all tasks within that workflow branch will execute in the specified environment.
57
+
58
+ Example:
59
+
60
+ .. code-block :: python
61
+
62
+ from pydra.environments import singularity
63
+ from pydra.compose import workflow, shell
64
+ from fileformats.generic import File
23
65
66
+ image = " /path/to/my_singularity_image.sif" # Replace with your Singularity image path
67
+
68
+ Singu = shell.define(" cat {file} " )
69
+
70
+ def MyWorkflow (file : File) -> str :
71
+ singu_task = workflow.add(
72
+ Singu(file = file ),
73
+ environment = singularity.Environment(image = image),
74
+ )
75
+ return singu_task.stdout
76
+
77
+ # Now you can use MyWorkflow, and the 'cat' task will run in the Singularity environment
24
78
25
79
26
80
Implementing new environment types
27
81
----------------------------------
28
82
29
- Work in progress...
83
+ Custom environment types can be implemented by creating a new class that inherits from `pydra.environments.Environment `.
84
+ These custom environment classes are typically located in the `pydra/environments/ ` directory.
85
+
86
+ Example (simplified custom environment):
87
+
88
+ .. code-block :: python
89
+
90
+ from pydra.environments import Environment as PydraEnvironment
91
+
92
+ class MyCustomEnvironment (PydraEnvironment ):
93
+ def __init__ (self , some_config : str ):
94
+ super ().__init__ ()
95
+ self .some_config = some_config
96
+
97
+ def _setup (self ):
98
+ # Logic to set up the custom environment
99
+ print (f " Setting up custom environment with config: { self .some_config} " )
100
+
101
+ def _execute (self , command : list ):
102
+ # Logic to execute a command within the custom environment
103
+ # This is where you would integrate with a custom execution system
104
+ print (f " Executing command: { ' ' .join(command)} in custom environment " )
105
+ # For demonstration, just return a dummy result
106
+ return {" stdout" : " Custom environment output" , " return_code" : 0 }
107
+
108
+ def _tear_down (self ):
109
+ # Logic to tear down the custom environment
110
+ print (" Tearing down custom environment" )
111
+
112
+ Then, you can use your custom environment like any other built-in environment:
113
+
114
+ .. code-block :: python
115
+
116
+ from pydra.compose import shell
117
+ # Assume MyCustomEnvironment is defined as above
118
+ my_task = shell.fuse(" echo Hello from custom env" )
119
+ outputs = my_task(environment = MyCustomEnvironment(some_config = " test" ))
120
+ print (outputs.stdout)
30
121
31
122
32
123
.. _Docker : https://www.docker.com/
0 commit comments