Update loops examples and sidebar layout (#1410)

**Pull Request Checklist**
- [x] 4 of 5 for #1408 
- [x] Documentation/examples added
- [x] [Good commit messages](https://cbea.ms/git-commit/) and/or PR
title

**Description of PR**
Improve the loops examples. Also change the examples sidebar layout to
lift up the sub folders to the top (i.e. removing the "idiomatic Hera
examples" drop down).

---------

Signed-off-by: Elliot Gunton <elliotgunton@gmail.com>

Author: elliotgunton

docs/examples/workflows-examples.md

@@ -1,20 +1,19 @@
 # Workflows Examples
 
-Hera has complete feature parity with YAML as a definition language for Argo Workflows. To demonstrate the equivalency
-(and improvements that Hera offers), we provide two collections of examples.
+Hera has complete feature parity with YAML as a definition language for Argo Workflows.
+
+The examples show off features and idiomatic usage of Hera such as the
+[Coinflip example](workflows/scripts/coinflip.md) recreated using the script decorator.
 
 The "Upstream" collection contains examples
 [directly from the Argo Workflows repository](https://github.com/argoproj/argo-workflows/tree/6e97c7d/examples), such as
 the [DAG Diamond example](workflows/upstream/dag_diamond.md), to demonstrate how the YAML spec maps to Hera classes.
-These examples are generated and compared in Hera's CI/CD pipeline to ensure that all the examples are possible to
-recreate in Hera.
+They may not be written as idiomatic Hera code (for example, using `Container` when a script decorator would be
+preferred).
 
 > If you'd like to contribute missing examples, please see the table below and follow the
 > [Contributing Guide](../CONTRIBUTING.md)!
 
-The "Hera" collection shows off features and idiomatic usage of Hera such as the
-[Coinflip example](workflows/scripts/coinflip.md) recreated using the script decorator.
-
 Explore the examples through the side bar!
 
 ## List of **missing** examples

…les/workflows/loops/script_basic_loop.md …examples/workflows/loops/basic_fanout.mddocs/examples/workflows/loops/script_basic_loop.md renamed to docs/examples/workflows/loops/basic_fanout.md docs/examples/workflows/loops/script_basic_loop.md renamed to docs/examples/workflows/loops/basic_fanout.md

@@ -1,8 +1,8 @@
-# Script Basic Loop
-
+# Basic Fanout
 
 
 
+This examples shows some basic hard-coded fan-outs.
 
 
 === "Hera"
@@ -36,11 +36,18 @@
                 ],
             )
 
-            # We can still pass a list of dict values to `with_items`, but must serialize them
+            # We can still pass a list of dict values to `with_items`, but must
+            # serialize them using Hera's `serialize` function
             print_message(
                 name="print-message-loop-with-items-list-of-dicts",
                 arguments={"message": "{{item}}"},
-                with_items=[serialize(item) for item in [{"my-key": "hello world"}, {"my-other-key": "goodbye world"}]],
+                with_items=[
+                    serialize(item)
+                    for item in [
+                        {"my-key": "hello world"},
+                        {"my-other-key": "goodbye world"},
+                    ]
+                ],
             )
     ```
 

docs/examples/workflows/loops/dynamic_fanout.md

@@ -2,8 +2,12 @@
 
 
 
-This example showcases how clients can use Hera to dynamically generate tasks that process outputs from one task in
-parallel. This is useful for batch jobs and instances where clients do not know ahead of time how many tasks/entities
+This examples shows how to dynamically fan out over a list of values.
+
+The fan-out task will consume the output of the previous task and process the list of values in parallel. We are
+printing to stdout to use the `result` output parameter, but generally you should use a named output parameter.
+
+Dynamic fan-outs are useful for batch jobs and instances where clients do not know ahead of time how many tasks/entities
 they may need to process.
 
 
@@ -16,19 +20,19 @@ they may need to process.
     @script()
     def generate():
         import json
+        import random
         import sys
 
-        # this can be anything! e.g fetch from some API, then in parallel process all entities; chunk database records
-        # and process them in parallel, etc.
-        json.dump([i for i in range(10)], sys.stdout)
+        # this can be anything! e.g fetch from some API, then in parallel process
+        # all entities; chunk database records and process them in parallel, etc.
+        json.dump([i for i in range(random.randint(8, 12))], sys.stdout)
 
 
     @script()
     def consume(value: int):
         print("Received value: {value}!".format(value=value))
 
 
-    # assumes you used `hera.set_global_token` and `hera.set_global_host` so that the workflow can be submitted
     with Workflow(generate_name="dynamic-fanout-", entrypoint="d") as w:
         with DAG(name="d"):
             g = generate()
@@ -67,8 +71,9 @@ they may need to process.
             import sys
             sys.path.append(os.getcwd())
             import json
+            import random
             import sys
-            json.dump([i for i in range(10)], sys.stdout)
+            json.dump([i for i in range(random.randint(8, 12))], sys.stdout)
           command:
           - python
       - name: consume

docs/examples/workflows/loops/dynamic_fanout_container.md

@@ -2,10 +2,12 @@
 
 
 
-This example showcases how clients can use Hera to dynamically generate tasks that process outputs from one task in
-parallel. Differ from dynamic_fanout.py, this example uses a container to generate the tasks and the dynamically
-created tasks are also container only.
-More details can be found here: https://github.com/argoproj-labs/hera-workflows/issues/250
+This examples shows how to dynamically fan out over a list of values using `Container` templates.
+
+The command `len=$((8 + RANDOM % 4)); json=$(seq 1 "$len" | paste -sd, -); echo "[$json]"` produces a list of random
+length (between 8 and 12), which is then echoed in json format. This matches the [dynamic fanout](dynamic-fanout.md)
+example that uses Python. In this example, we must specify the arguments as `Container` does not automatically match
+them for us.
 
 
 === "Hera"
@@ -16,7 +18,7 @@ More details can be found here: https://github.com/argoproj-labs/hera-workflows/
     generate = Container(
         name="generate",
         image="alpine:latest",
-        command=["echo", '[{"value": "a"}, {"value": "b"}, {"value": "c"}]'],
+        command=['len=$((8 + RANDOM % 4)); json=$(seq 1 "$len" | paste -sd, -); echo "[$json]"'],
     )
 
     fanout = Container(
@@ -26,15 +28,13 @@ More details can be found here: https://github.com/argoproj-labs/hera-workflows/
         command=["echo", "{{inputs.parameters.value}}"],
     )
 
-    # assumes you used `hera.set_global_token` and `hera.set_global_host` so that the workflow can be submitted
-    with Workflow(generate_name="dynamic-fanout-container-", entrypoint="d") as w:
+    with Workflow(
+        generate_name="dynamic-fanout-container-",
+        entrypoint="d",
+    ) as w:
         with DAG(name="d"):
-            # this can be anything! e.g. fetch from some API, then in parallel process all entities; chunk database records
-            # and process them in parallel, etc.
             g = generate()
-            f = fanout(
-                arguments={"value": "{{item.value}}"}, with_param=g.result
-            )  # this make the task fan out over the `with_param`
+            f = fanout(arguments={"value": "{{item}}"}, with_param=g.result)
             g >> f
     ```
 
@@ -60,13 +60,12 @@ More details can be found here: https://github.com/argoproj-labs/hera-workflows/
             arguments:
               parameters:
               - name: value
-                value: '{{item.value}}'
+                value: '{{item}}'
       - name: generate
         container:
           image: alpine:latest
           command:
-          - echo
-          - '[{"value": "a"}, {"value": "b"}, {"value": "c"}]'
+          - len=$((8 + RANDOM % 4)); json=$(seq 1 "$len" | paste -sd, -); echo "[$json]"
       - name: fanout
         container:
           image: alpine:latest

…/workflows/loops/dynamic_fanout_fanin.md docs/examples/workflows/loops/fan_in.mddocs/examples/workflows/loops/dynamic_fanout_fanin.md renamed to docs/examples/workflows/loops/fan_in.md docs/examples/workflows/loops/dynamic_fanout_fanin.md renamed to docs/examples/workflows/loops/fan_in.md

@@ -1,8 +1,10 @@
-# Dynamic Fanout Fanin
+# Fan In
 
 
 
+This example shows how to collect values in a "fan-in" task after the fan-out.
 
+This also works for the `result` output parameter (as long as nothing else is in stdout!).
 
 
 === "Hera"
@@ -20,11 +22,18 @@
         json.dump([{"value": i} for i in range(10)], sys.stdout)
 
 
-    @script(outputs=[Parameter(name="value", value_from=ValueFrom(path="/tmp/value"))])
-    def fanout(object: dict):
-        print("Received object: {object}!".format(object=object))
-        # Output the content of the "value" key in the object
-        value = object["value"]
+    @script(
+        outputs=[
+            Parameter(
+                name="value",
+                value_from=ValueFrom(path="/tmp/value"),
+            )
+        ]
+    )
+    def fanout(my_dict: dict):
+        print("Received object: {my_dict}!".format(my_dict=my_dict))
+        # Output the content of the "value" key in the dict
+        value = my_dict["value"]
         with open("/tmp/value", "w") as f:
             f.write(str(value))
 
@@ -34,12 +43,11 @@
         print("Received values: {values}!".format(values=values))
 
 
-    # assumes you used `hera.set_global_token` and `hera.set_global_host` so that the workflow can be submitted
-    with Workflow(generate_name="dynamic-fanout-fanin", entrypoint="d") as w:
+    with Workflow(generate_name="fan-in-", entrypoint="d") as w:
         with DAG(name="d"):
             g = generate()
             fout = fanout(with_param=g.result)
-            fin = fanin(arguments=fout.get_parameter("value").with_name("values"))
+            fin = fanin(arguments={"values": fout.get_parameter("value")})
             g >> fout >> fin
     ```
 
@@ -49,7 +57,7 @@
     apiVersion: argoproj.io/v1alpha1
     kind: Workflow
     metadata:
-      generateName: dynamic-fanout-fanin
+      generateName: fan-in-
     spec:
       entrypoint: d
       templates:
@@ -64,7 +72,7 @@
             withParam: '{{tasks.generate.outputs.result}}'
             arguments:
               parameters:
-              - name: object
+              - name: my_dict
                 value: '{{item}}'
           - name: fanin
             depends: fanout
@@ -88,7 +96,7 @@
       - name: fanout
         inputs:
           parameters:
-          - name: object
+          - name: my_dict
         outputs:
           parameters:
           - name: value
@@ -101,11 +109,11 @@
             import sys
             sys.path.append(os.getcwd())
             import json
-            try: object = json.loads(r'''{{inputs.parameters.object}}''')
-            except: object = r'''{{inputs.parameters.object}}'''
+            try: my_dict = json.loads(r'''{{inputs.parameters.my_dict}}''')
+            except: my_dict = r'''{{inputs.parameters.my_dict}}'''
 
-            print('Received object: {object}!'.format(object=object))
-            value = object['value']
+            print('Received object: {my_dict}!'.format(my_dict=my_dict))
+            value = my_dict['value']
             with open('/tmp/value', 'w') as f:
                 f.write(str(value))
           command:

…ows/loops/dynamic_fanout_extra_kwargs.md …s/workflows/loops/fanout_extra_kwargs.mddocs/examples/workflows/loops/dynamic_fanout_extra_kwargs.md renamed to docs/examples/workflows/loops/fanout_extra_kwargs.md docs/examples/workflows/loops/dynamic_fanout_extra_kwargs.md renamed to docs/examples/workflows/loops/fanout_extra_kwargs.md

@@ -1,11 +1,8 @@
-# Dynamic Fanout Extra Kwargs
+# Fanout Extra Kwargs
 
 
 
-This example showcases how clients can use Hera to dynamically generate tasks that process outputs from one task in
-parallel. This is useful for batch jobs and instances where clients do not know ahead of time how many tasks/entities
-they may need to process. In addition to the fanout, this example showcases how one can set up extra parameters for
-the job to dictate what the fanout should execute over.
+This example shows how to use a fan-out over one argument, while keeping the others the same.
 
 
 === "Hera"
@@ -19,8 +16,8 @@ the job to dictate what the fanout should execute over.
         import json
         import sys
 
-        # this can be anything! e.g fetch from some API, then in parallel process all entities; chunk database records
-        # and process them in parallel, etc.
+        # this can be anything! e.g fetch from some API, then in parallel process
+        # all entities; chunk database records and process them in parallel, etc.
         json.dump([i for i in range(10)], sys.stdout)
 
 
@@ -35,23 +32,34 @@ the job to dictate what the fanout should execute over.
         )
 
 
-    # assumes you used `hera.set_global_token` and `hera.set_global_host` so that the workflow can be submitted
     with Workflow(generate_name="dynamic-fanout-", entrypoint="d") as w:
         with DAG(name="d"):
             g = generate()
-            # the following fanout will occur over the items in the list that is returned from the generate script
-            # the `extra_param1` will take the `hello world` value while `extra_param2` will hold the default value of 42
-            c1 = consume(name="c1", with_param=g.result, arguments={"value": "{{item}}", "extra_param1": "hello world"})
 
-            # the following fanout will occur over the items in the list that is returned from the generate script
-            # the `extra_param1` will take the `hello world` value while `extra_param2` will hold the default value of 123
+            # We use `value` to fan-out over the values from `generate`, while the
+            # other arguments remain the same for all fan-out tasks.
+            # `extra_param1` is set here, while `extra_param1` has a default
+            # value of 42 in the script
+            c1 = consume(
+                name="c1",
+                with_param=g.result,
+                arguments={
+                    "value": "{{item}}",
+                    "extra_param1": "hello world",
+                },
+            )
+
+            # Here is the same fan-out, except we are also setting `extra_param2`
             c2 = consume(
                 name="c2",
                 with_param=g.result,
-                arguments={"value": "{{item}}", "extra_param1": "hello world", "extra_param2": "123"},
+                arguments={
+                    "value": "{{item}}",
+                    "extra_param1": "hello world",
+                    "extra_param2": "123",
+                },
             )
-            g >> c1
-            g >> c2
+            g >> [c1, c2]
     ```
 
 === "YAML"

…ows/loops/dynamic_fanout_json_payload.md …s/workflows/loops/json_payload_fanout.mddocs/examples/workflows/loops/dynamic_fanout_json_payload.md renamed to docs/examples/workflows/loops/json_payload_fanout.md docs/examples/workflows/loops/dynamic_fanout_json_payload.md renamed to docs/examples/workflows/loops/json_payload_fanout.md

@@ -1,10 +1,8 @@
-# Dynamic Fanout Json Payload
+# Json Payload Fanout
 
 
 
-This example showcases how clients can use Hera to dynamically generate tasks that process outputs from one task in
-parallel. This is useful for batch jobs and instances where clients do not know ahead of time how many tasks/entities
-they may need to process. The fanout occurs over independent JSON payloads coming from the generate script
+This example shows how you can fan-out over a JSON payload (a JSON list of dicts), and let Hera match the arguments for you.
 
 
 === "Hera"
@@ -18,8 +16,8 @@ they may need to process. The fanout occurs over independent JSON payloads comin
         import json
         import sys
 
-        # this can be anything! e.g fetch from some API, then in parallel process all entities; chunk database records
-        # and process them in parallel, etc.
+        # this can be anything! e.g fetch from some API, then in parallel process
+        # all entities; chunk database records and process them in parallel, etc.
         json.dump([{"p1": i + 1, "p2": i + 2, "p3": i + 3} for i in range(10)], sys.stdout)
 
 
@@ -28,8 +26,7 @@ they may need to process. The fanout occurs over independent JSON payloads comin
         print("Received p1={p1}, p2={p2}, p3={p3}".format(p1=p1, p2=p2, p3=p3))
 
 
-    # assumes you used `hera.set_global_token` and `hera.set_global_host` so that the workflow can be submitted
-    with Workflow(generate_name="dynamic-fanout-", entrypoint="d") as w:
+    with Workflow(generate_name="json-payload-fanout-", entrypoint="d") as w:
         with DAG(name="d"):
             g = generate()
             c = consume(with_param=g.result)
@@ -42,7 +39,7 @@ they may need to process. The fanout occurs over independent JSON payloads comin
     apiVersion: argoproj.io/v1alpha1
     kind: Workflow
     metadata:
-      generateName: dynamic-fanout-
+      generateName: json-payload-fanout-
     spec:
       entrypoint: d
       templates: