Skip to content

Commit 3b30d5b

Browse files
authored
Update experimental examples (#1411)
**Pull Request Checklist** - [x] 5 of 5. Closes #1408. Fixes #1403 - [x] Tests added - [x] Documentation/examples added - [x] [Good commit messages](https://cbea.ms/git-commit/) and/or PR title **Description of PR** * Add intro text * Use Workflow throughout * Ensure they are all runnable (except for template ref example which requires setting up the WorkflowTemplates) --------- Signed-off-by: Elliot Gunton <elliotgunton@gmail.com>
1 parent 19363f6 commit 3b30d5b

26 files changed

+299
-175
lines changed

docs/examples/workflows/experimental/new_container_decorator.md

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33

44

5-
5+
This example shows the use of the container decorator and special Input/Output classes.
66

77

88
=== "Hera"
@@ -11,20 +11,21 @@
1111
from typing_extensions import Annotated
1212

1313
from hera.shared import global_config
14-
from hera.workflows import Input, Output, Parameter, WorkflowTemplate
14+
from hera.workflows import Input, Output, Parameter, Workflow
1515

1616
global_config.experimental_features["decorator_syntax"] = True
1717

1818

19-
# We start by defining our Workflow Template
20-
w = WorkflowTemplate(name="my-template")
19+
# We start by defining our Workflow
20+
w = Workflow(generate_name="container-workflow-")
2121

2222

2323
# This defines the template's inputs
2424
class MyInput(Input):
2525
user: str = "Hera"
2626

2727

28+
# This defines the template's outputs
2829
class MyOutput(Output):
2930
container_greeting: Annotated[
3031
str,
@@ -35,6 +36,8 @@
3536
]
3637

3738

39+
# We then use the decorators of the `Workflow` object
40+
# to set the entrypoint and create a Container template
3841
@w.set_entrypoint
3942
@w.container(
4043
image="busybox",
@@ -48,9 +51,9 @@
4851

4952
```yaml linenums="1"
5053
apiVersion: argoproj.io/v1alpha1
51-
kind: WorkflowTemplate
54+
kind: Workflow
5255
metadata:
53-
name: my-template
56+
generateName: container-workflow-
5457
spec:
5558
entrypoint: basic-hello-world
5659
templates:

docs/examples/workflows/experimental/new_dag_decorator_artifacts.md

Lines changed: 56 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,10 @@
22

33

44

5+
This example shows how to pass Artifacts between scripts in a dag, using the new decorators.
56

7+
The DAG decorator function can easily lift out an output artifact from a task as an output of the
8+
DAG itself by referencing it in an `Output` class.
69

710

811
=== "Hera"
@@ -11,21 +14,33 @@
1114
from typing_extensions import Annotated
1215

1316
from hera.shared import global_config
14-
from hera.workflows import Artifact, ArtifactLoader, Input, Output, Workflow
17+
from hera.workflows import (
18+
Artifact,
19+
ArtifactLoader,
20+
Input,
21+
NoneArchiveStrategy,
22+
Output,
23+
Workflow,
24+
)
1525

1626
global_config.experimental_features["decorator_syntax"] = True
1727

1828

19-
w = Workflow(generate_name="my-workflow-")
29+
w = Workflow(generate_name="artifact-workflow-")
2030

2131

2232
class ArtifactOutput(Output):
23-
an_artifact: Annotated[str, Artifact(name="an-artifact")]
33+
an_artifact: Annotated[str, Artifact(name="an-artifact", archive=NoneArchiveStrategy())]
2434

2535

2636
class ConcatInput(Input):
27-
word_a: Annotated[str, Artifact(name="word_a", loader=ArtifactLoader.json)]
28-
word_b: Annotated[str, Artifact(name="word_b", loader=ArtifactLoader.json)]
37+
word_a: Annotated[str, Artifact(name="word_a", loader=ArtifactLoader.file)]
38+
word_b: Annotated[str, Artifact(name="word_b", loader=ArtifactLoader.file)]
39+
40+
41+
@w.script()
42+
def create_artifact() -> ArtifactOutput:
43+
return ArtifactOutput(an_artifact="hello world")
2944

3045

3146
@w.script()
@@ -40,11 +55,12 @@
4055

4156
@w.set_entrypoint
4257
@w.dag()
43-
def worker(worker_input: WorkerInput) -> ArtifactOutput:
58+
def worker() -> ArtifactOutput:
59+
create = create_artifact()
4460
concat_1 = concat(
4561
ConcatInput(
46-
word_a=worker_input.artifact_a,
47-
word_b=worker_input.artifact_b,
62+
word_a=create.an_artifact,
63+
word_b=create.an_artifact,
4864
)
4965
)
5066

@@ -65,10 +81,32 @@
6581
apiVersion: argoproj.io/v1alpha1
6682
kind: Workflow
6783
metadata:
68-
generateName: my-workflow-
84+
generateName: artifact-workflow-
6985
spec:
7086
entrypoint: worker
7187
templates:
88+
- name: create-artifact
89+
outputs:
90+
artifacts:
91+
- name: an-artifact
92+
path: /tmp/hera-outputs/artifacts/an-artifact
93+
archive:
94+
none: {}
95+
script:
96+
image: python:3.9
97+
source: '{{inputs.parameters}}'
98+
args:
99+
- -m
100+
- hera.workflows.runner
101+
- -e
102+
- examples.workflows.experimental.new_dag_decorator_artifacts:create_artifact
103+
command:
104+
- python
105+
env:
106+
- name: hera__outputs_directory
107+
value: /tmp/hera-outputs
108+
- name: hera__script_pydantic_io
109+
value: ''
72110
- name: concat
73111
inputs:
74112
artifacts:
@@ -80,6 +118,8 @@
80118
artifacts:
81119
- name: an-artifact
82120
path: /tmp/hera-outputs/artifacts/an-artifact
121+
archive:
122+
none: {}
83123
script:
84124
image: python:3.9
85125
source: '{{inputs.parameters}}'
@@ -98,14 +138,17 @@
98138
- name: worker
99139
dag:
100140
tasks:
141+
- name: create
142+
template: create-artifact
101143
- name: concat-1
144+
depends: create
102145
template: concat
103146
arguments:
104147
artifacts:
105148
- name: word_a
106-
from: '{{inputs.artifacts.artifact_a}}'
149+
from: '{{tasks.create.outputs.artifacts.an-artifact}}'
107150
- name: word_b
108-
from: '{{inputs.artifacts.artifact_b}}'
151+
from: '{{tasks.create.outputs.artifacts.an-artifact}}'
109152
- name: concat-2-custom-name
110153
depends: concat-1
111154
template: concat
@@ -115,13 +158,11 @@
115158
from: '{{tasks.concat-1.outputs.artifacts.an-artifact}}'
116159
- name: word_b
117160
from: '{{tasks.concat-1.outputs.artifacts.an-artifact}}'
118-
inputs:
119-
artifacts:
120-
- name: artifact_a
121-
- name: artifact_b
122161
outputs:
123162
artifacts:
124163
- name: an-artifact
125164
from: '{{tasks.concat-2-custom-name.outputs.artifacts.an-artifact}}'
165+
archive:
166+
none: {}
126167
```
127168

docs/examples/workflows/experimental/new_dag_decorator_inner_dag.md

Lines changed: 10 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33

44

5-
5+
This example shows how to run an inner DAG within another DAG.
66

77

88
=== "Hera"
@@ -14,7 +14,7 @@
1414
global_config.experimental_features["decorator_syntax"] = True
1515

1616

17-
w = Workflow(generate_name="my-workflow-")
17+
w = Workflow(generate_name="inner-dag-workflow-")
1818

1919

2020
class SetupOutput(Output):
@@ -57,9 +57,9 @@
5757

5858
@w.set_entrypoint
5959
@w.dag()
60-
def outer_dag(worker_input: WorkerInput) -> WorkerOutput:
61-
sub_dag_a = worker(WorkerInput(value_a="dag_a", value_b=worker_input.value_a))
62-
sub_dag_b = worker(WorkerInput(value_a="dag_b", value_b=worker_input.value_b))
60+
def outer_dag() -> WorkerOutput:
61+
sub_dag_a = worker(WorkerInput(value_a="dag_a1", value_b="dag_a2"))
62+
sub_dag_b = worker(WorkerInput(value_a="dag_b1", value_b="dag_b2"))
6363

6464
sub_dag_c = worker(WorkerInput(value_a=sub_dag_a.value, value_b=sub_dag_b.value))
6565

@@ -72,7 +72,7 @@
7272
apiVersion: argoproj.io/v1alpha1
7373
kind: Workflow
7474
metadata:
75-
generateName: my-workflow-
75+
generateName: inner-dag-workflow-
7676
spec:
7777
entrypoint: outer-dag
7878
templates:
@@ -166,17 +166,17 @@
166166
arguments:
167167
parameters:
168168
- name: value_a
169-
value: dag_a
169+
value: dag_a1
170170
- name: value_b
171-
value: '{{inputs.parameters.value_a}}'
171+
value: dag_a2
172172
- name: sub-dag-b
173173
template: worker
174174
arguments:
175175
parameters:
176176
- name: value_a
177-
value: dag_b
177+
value: dag_b1
178178
- name: value_b
179-
value: '{{inputs.parameters.value_b}}'
179+
value: dag_b2
180180
- name: sub-dag-c
181181
depends: sub-dag-a && sub-dag-b
182182
template: worker
@@ -186,10 +186,6 @@
186186
value: '{{tasks.sub-dag-a.outputs.parameters.value}}'
187187
- name: value_b
188188
value: '{{tasks.sub-dag-b.outputs.parameters.value}}'
189-
inputs:
190-
parameters:
191-
- name: value_a
192-
- name: value_b
193189
outputs:
194190
parameters:
195191
- name: value

docs/examples/workflows/experimental/new_dag_decorator_params.md

Lines changed: 22 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33

44

5-
5+
This example shows how parameters can be passed into, within and out of a DAG.
66

77

88
=== "Hera"
@@ -17,7 +17,12 @@
1717
global_config.experimental_features["decorator_syntax"] = True
1818

1919

20-
w = Workflow(generate_name="my-workflow-")
20+
# Create a Workflow - we pass arguments here for the
21+
# entrypoint (designated by the `set_entrypoint` decorator)
22+
w = Workflow(
23+
generate_name="parameters-workflow-",
24+
arguments={"value_b": "a value for b!"},
25+
)
2126

2227

2328
class SetupConfig(BaseModel):
@@ -26,8 +31,10 @@
2631

2732
class SetupOutput(Output):
2833
environment_parameter: str
29-
an_annotated_parameter: Annotated[int, Parameter()] # use an annotated non-str, infer name from field
30-
setup_config: Annotated[SetupConfig, Parameter(name="setup-config")] # use a pydantic BaseModel
34+
an_annotated_parameter: Annotated[int, Parameter(description="infer name from field")]
35+
setup_config: Annotated[
36+
SetupConfig, Parameter(name="setup-config")
37+
] # a Pydantic BaseModel can be a single input Parameter
3138

3239

3340
@w.script()
@@ -53,7 +60,7 @@
5360
@w.script()
5461
def concat(concat_input: ConcatInput) -> Output:
5562
res = f"{concat_input.word_a} {concat_input.word_b}"
56-
if concat_input.reverse:
63+
if concat_input.concat_config.reverse:
5764
res = res[::-1]
5865
return Output(result=res)
5966

@@ -88,7 +95,10 @@
8895
task_b = concat(ConcatInput(word_a=worker_input.value_b, word_b=setup_task.result))
8996
final_task = concat(ConcatInput(word_a=task_a.result, word_b=task_b.result))
9097

91-
return WorkerOutput(result_value=final_task.result, another_value=setup_task.an_annotated_parameter)
98+
return WorkerOutput(
99+
result_value=final_task.result,
100+
another_value=setup_task.an_annotated_parameter,
101+
)
92102
```
93103

94104
=== "YAML"
@@ -97,7 +107,7 @@
97107
apiVersion: argoproj.io/v1alpha1
98108
kind: Workflow
99109
metadata:
100-
generateName: my-workflow-
110+
generateName: parameters-workflow-
101111
spec:
102112
entrypoint: worker
103113
templates:
@@ -108,6 +118,7 @@
108118
valueFrom:
109119
path: /tmp/hera-outputs/parameters/environment_parameter
110120
- name: an_annotated_parameter
121+
description: infer name from field
111122
valueFrom:
112123
path: /tmp/hera-outputs/parameters/an_annotated_parameter
113124
- name: setup-config
@@ -206,5 +217,9 @@
206217
- name: another_value
207218
valueFrom:
208219
parameter: '{{tasks.setup-task.outputs.parameters.an_annotated_parameter}}'
220+
arguments:
221+
parameters:
222+
- name: value_b
223+
value: a value for b!
209224
```
210225

docs/examples/workflows/experimental/new_decorators_auto_template_refs.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,9 @@
22

33

44

5+
This example shows how a Workflow can reference WorkflowTemplates.
56

7+
Note this example will not run unless you create the WorkflowTemplates first.
68

79

810
=== "Hera"

0 commit comments

Comments
 (0)