Refactor/standalone design (#41)

* add quicksilver
* save state of refactored app/network

This adds a shared template used for networking and app.
I think I can simplify this further to have a common shared
template.

* refactor to add common design jobset package

If this works it could be really cool - we have a jobs package that defines
common jobset (metric set) interfaces that can be shared between standalone
apps. As an example, the launcher/worker design works for both network
and traditional apps, with slight differences that are determined based
on the struct that inherits the interface. I hope this works!

* add pennant
* add ensure default name to success jobs
* simplify entrypoint script logic further

the common prefix to generate the hosts lists can be shared between
applications. We can also define it to be a more general prefix
field because it does not necessarily need to be mpirun

* quicksilver can use common prefix too
* also simplify storage interfaces!

This is so satisfying!

Signed-off-by: vsoch <[email protected]>

Author: vsoch

.github/workflows/main.yaml

@@ -70,7 +70,9 @@ jobs:
                ["io-host-volume", "ghcr.io/converged-computing/metric-sysstat:latest", 60],                 # storage test
                ["io-fio", "ghcr.io/converged-computing/metric-fio:latest", 120],                            # storage test
                ["app-amg", "ghcr.io/converged-computing/metric-amg:latest", 120],                           # standalone app test
-               ["app-kripke", "ghcr.io/converged-computing/metric-amg:latest", 120],                        # standalone app test
+               ["app-kripke", "ghcr.io/converged-computing/metric-kripke:latest", 120],                     # standalone app test
+               ["app-pennant", "ghcr.io/converged-computing/metric-pennant:latest", 120],                   # standalone app test
+               ["app-quicksilver", "ghcr.io/converged-computing/metric-quicksilver:latest", 120],           # standalone app test
                ["app-lammps", "ghcr.io/converged-computing/metric-lammps:latest", 120]]                     # standalone app test
 
     steps:

README.md

@@ -18,9 +18,11 @@ To learn more:
 - For services we are measuring, we likely need to be able to kill after N seconds (to complete job) or to specify the success policy on the metrics containers instead of the application
 - Look into pod affinity/anti-affintiy vs. topology constraint (which do we want)?
 - Add assertions checking for python tests
-- Plotting examples needed for
+- Plotting examples (python parsers) needed for
   - io-sysstat
   - app-kripke
+  - app-quicksilver
+  - app-pennant
 
 ## License
 

docs/_static/data/metrics.json

@@ -20,6 +20,20 @@
   "image": "ghcr.io/converged-computing/metric-lammps:latest",
   "url": "https://www.lammps.org/"
  },
+ {
+  "name": "app-pennant",
+  "description": "Unstructured mesh hydrodynamics for advanced architectures ",
+  "type": "standalone",
+  "image": "ghcr.io/converged-computing/metric-pennant:latest",
+  "url": "https://github.com/LLNL/pennant"
+ },
+ {
+  "name": "app-quicksilver",
+  "description": "A proxy app for the Monte Carlo Transport Code",
+  "type": "standalone",
+  "image": "ghcr.io/converged-computing/metric-quicksilver:latest",
+  "url": "https://github.com/LLNL/Quicksilver"
+ },
  {
   "name": "io-fio",
   "description": "Flexible IO Tester (FIO)",

docs/getting_started/metrics.md

@@ -7,7 +7,7 @@ The following metrics are under development (or being planned).
  - [Application Metrics](https://converged-computing.github.io/metrics-operator/getting_started/metrics.html#application)
  - [Standalone Metrics](https://converged-computing.github.io/metrics-operator/getting_started/metrics.html#standalone)
 
-<iframe src="../_static/data/table.html" style="width:100%; height:650px;" frameBorder="0"></iframe>
+<iframe src="../_static/data/table.html" style="width:100%; height:700px;" frameBorder="0"></iframe>
 
 All metrics can be customized with the following variables
 
@@ -290,6 +290,169 @@ More likely you want an actual problem size on a specific number of node and tas
 run a larger problem and the parser does not work as expected, please [send us the output](https://github.com/converged-computing/metrics-operator/issues) and we will provide an updated parser.
 See [this guide](https://asc.llnl.gov/sites/asc/files/2020-09/AMG_Summary_v1_7.pdf) for more detail.
 
+
+#### app-quicksilver
+
+ - [Standalone Metric Set](user-guide.md#application-metric-set)
+ - *[app-quicksilver](https://github.com/converged-computing/metrics-operator/tree/main/examples/tests/app-quicksilver)*
+
+Quicksilver is a proxy app for Monte Carlo simulation code. You can learn more about it on the [GitHub repository](https://github.com/LLNL/Quicksilver/).
+By default, akin to other apps we expose the entire mpirun command along with the working directory for you to adjust.
+
+| Name | Description | Option Key | Type | Default |
+|-----|-------------|------------|------|---------|
+| command | The qs command (without mpirun) | options->command |string | (see below) |
+| mpirun | The mpirun command (and arguments) | options->mpirun | string | (see below) | 
+| workdir | The working directory for the command | options->workdir | string | /opt/AMG |
+
+By default, when not set, you will just run the qs (quicksilver) binary on a sample problem, represented by an input text file: 
+
+```bash
+# mpirun
+mpirun --hostfile ./hostlist.txt
+
+# command
+qs /opt/quicksilver/Examples/CORAL2_Benchmark/Problem1/Coral2_P1.inp
+
+# Assembled into problem.sh as follows:
+mpirun --hostfile ./hostlist.txt ./problem.sh
+```
+
+There are many problems that come in the container, and here are the fullpaths:
+
+```console
+# Example command
+qs /opt/quicksilver/Examples/CORAL2_Benchmark/Problem1/Coral2_P1.inp
+
+# All examples:
+/opt/quicksilver/Examples/AllScattering/scatteringOnly.inp
+/opt/quicksilver/Examples/NoCollisions/no.collisions.inp
+/opt/quicksilver/Examples/NonFlatXC/NonFlatXC.inp
+/opt/quicksilver/Examples/CORAL2_Benchmark/Problem2/Coral2_P2_4096.inp
+/opt/quicksilver/Examples/CORAL2_Benchmark/Problem2/Coral2_P2.inp
+/opt/quicksilver/Examples/CORAL2_Benchmark/Problem2/Coral2_P2_1.inp
+/opt/quicksilver/Examples/CORAL2_Benchmark/Problem1/Coral2_P1.inp
+/opt/quicksilver/Examples/CORAL2_Benchmark/Problem1/Coral2_P1_1.inp
+/opt/quicksilver/Examples/CORAL2_Benchmark/Problem1/Coral2_P1_4096.inp
+/opt/quicksilver/Examples/CTS2_Benchmark/CTS2.inp
+/opt/quicksilver/Examples/CTS2_Benchmark/CTS2_36.inp
+/opt/quicksilver/Examples/CTS2_Benchmark/CTS2_1.inp
+/opt/quicksilver/Examples/AllAbsorb/allAbsorb.inp
+/opt/quicksilver/Examples/Homogeneous/homogeneousProblem_v4_ts.inp
+/opt/quicksilver/Examples/Homogeneous/homogeneousProblem_v5_ts.inp
+/opt/quicksilver/Examples/Homogeneous/homogeneousProblem.inp
+/opt/quicksilver/Examples/Homogeneous/homogeneousProblem_v3_wq.inp
+/opt/quicksilver/Examples/Homogeneous/homogeneousProblem_v7_ts.inp
+/opt/quicksilver/Examples/Homogeneous/homogeneousProblem_v4_tm.inp
+/opt/quicksilver/Examples/Homogeneous/homogeneousProblem_v3.inp
+/opt/quicksilver/Examples/AllEscape/allEscape.inp
+/opt/quicksilver/Examples/NoFission/noFission.inp
+```
+
+You can also look more closely in the [GitHub repository](https://github.com/LLNL/Quicksilver/tree/master/Examples).
+
+#### app-pennant
+
+ - [Standalone Metric Set](user-guide.md#application-metric-set)
+ - *[app-pennant](https://github.com/converged-computing/metrics-operator/tree/main/examples/tests/app-pennant)*
+
+Pennant is an unstructured mesh hydrodynamics for advanced architectures. The documentation is sparse, but you
+can find the [source code on GitHub](https://github.com/llnl/pennant). 
+By default, akin to other apps we expose the entire mpirun prefix and command along with the working directory for you to adjust.
+
+| Name | Description | Option Key | Type | Default |
+|-----|-------------|------------|------|---------|
+| command | The pennant command (without mpirun) | options->command |string | (see below) |
+| mpirun | The mpirun command (and arguments) | options->mpirun | string | (see below) | 
+| workdir | The working directory for the command | options->workdir | string | /opt/AMG |
+
+By default, when not set, you will just run pennant on a test problem, represented by an input text file: 
+
+```bash
+# mpirun
+mpirun --hostfile ./hostlist.txt
+
+# command
+pennant /opt/pennant/test/sedovsmall/sedovsmall.pnt
+
+# Assembled into problem.sh as follows:
+mpirun --hostfile ./hostlist.txt ./problem.sh
+```
+
+There are many input files that come in the container, and here are the fullpaths in `/opt/pennant/test`:
+
+<details>
+
+<summary>Input files available to pennant</summary>
+
+```console
+|-- leblanc
+|   |-- leblanc.pnt
+|   |-- leblanc.xy.std
+|   `-- leblanc.xy.std4
+|-- leblancbig
+|   `-- leblancbig.pnt
+|-- leblancx16
+|   `-- leblancx16.pnt
+|-- leblancx4
+|   `-- leblancx4.pnt
+|-- leblancx48
+|   `-- leblancx48.pnt
+|-- leblancx64
+|   `-- leblancx64.pnt
+|-- noh
+|   |-- noh.pnt
+|   |-- noh.xy.std
+|   `-- noh.xy.std4
+|-- nohpoly
+|   `-- nohpoly.pnt
+|-- nohsmall
+|   |-- nohsmall.pnt
+|   |-- nohsmall.xy.std
+|   `-- nohsmall.xy.std4
+|-- nohsquare
+|   `-- nohsquare.pnt
+|-- sample_outputs
+|   |-- edison
+|   |   |-- leblancbig.thr1.out
+|   |   |-- leblancx16.thr1024.out
+|   |   |-- leblancx4.thr16.out
+|   |   |-- leblancx64.mpi2048.out
+|   |   `-- nohpoly.thr1.out
+|   `-- vulcan
+|       |-- leblancx16.out
+|       |-- leblancx48.out
+|       |-- sedovflat.out
+|       |-- sedovflatx16.out
+|       |-- sedovflatx4.out
+|       `-- sedovflatx40.out
+|-- sedov
+|   |-- sedov.pnt
+|   |-- sedov.xy.std
+|   `-- sedov.xy.std4
+|-- sedovbig
+|   `-- sedovbig.pnt
+|-- sedovflat
+|   `-- sedovflat.pnt
+|-- sedovflatx120
+|   `-- sedovflatx120.pnt
+|-- sedovflatx16
+|   `-- sedovflatx16.pnt
+|-- sedovflatx4
+|   `-- sedovflatx4.pnt
+|-- sedovflatx40
+|   `-- sedovflatx40.pnt
+`-- sedovsmall
+    |-- sedovsmall.pnt
+    |-- sedovsmall.xy
+    |-- sedovsmall.xy.std
+    `-- sedovsmall.xy.std4
+```
+
+</details>
+
+And likely you will need to adjust the mpirun parameters, etc.
+
 #### app-kripke
 
  - [Standalone Metric Set](user-guide.md#application-metric-set)

examples/tests/app-pennant/README.md

@@ -0,0 +1,143 @@
+# Pennant Example
+
+This is an example of a metric app, Pennant, which is part of the [coral 2 benchmarks](https://asc.llnl.gov/coral-2-benchmarks). 
+We have not yet added a Python example as we want a use case first, but can and will when it is warranted.
+
+## Usage
+
+Create a cluster
+
+```bash
+kind create cluster
+```
+
+and install JobSet to it.
+
+```bash
+VERSION=v0.2.0
+kubectl apply --server-side -f https://github.com/kubernetes-sigs/jobset/releases/download/$VERSION/manifests.yaml
+```
+
+Install the operator (from the development manifest here):
+
+```bash
+$ kubectl apply -f ../../dist/metrics-operator-dev.yaml
+```
+
+How to see metrics operator logs:
+
+```bash
+$ kubectl logs -n metrics-system metrics-controller-manager-859c66464c-7rpbw
+```
+
+Then create the metrics set. This is going to run a single run of LAMMPS over MPI!
+as lammps runs.
+
+```bash
+kubectl apply -f metrics.yaml
+```
+
+Wait until you see pods created by the job and then running (there should be two - a launcher and worker for LAMMPS):
+
+```bash
+kubectl get pods
+```
+```diff
+NAME                           READY   STATUS    RESTARTS   AGE
+metricset-sample-l-0-0-lt782   1/1     Running   0          3s
+metricset-sample-w-0-0-4s5p9   1/1     Running   0          3s
+```
+
+In the above, "l" is a launcher pod, and "w" is a worker node.
+If you inspect the log for the launcher you'll see a short sleep (the network isn't up immediately)
+and then the example running, and the log is printed to the console.
+
+```bash
+kubectl logs metricset-sample-l-0-0-lt782 -f
+```
+```console
+METADATA START {"pods":2,"completions":2,"metricName":"app-pennant","metricDescription":"Unstructured mesh hydrodynamics for advanced architectures ","metricType":"standalone","metricOptions":{"command":"pennant /opt/pennant/test/sedovsmall/sedovsmall.pnt","completions":0,"mpirun":"mpirun --hostfile ./hostlist.txt","rate":10,"workdir":"/opt/pennant/test"}}
+METADATA END
+Sleeping for 10 seconds waiting for network...
+METRICS OPERATOR COLLECTION START
+METRICS OPERATOR TIMEPOINT
+********************
+Running PENNANT v0.9
+********************
+
+Running on 2 MPI PE(s)
+Running on 8 thread(s)
+--- Mesh Information ---
+Points:  100
+Zones:  81
+Sides:  324
+Edges:  189
+Side chunks:  21
+Point chunks:  8
+Zone chunks:  6
+Chunk size:  16
+------------------------
+Energy check:  total energy  =   2.467991e-01
+(internal =   2.467991e-01, kinetic =   0.000000e+00)
+End cycle      1, time = 2.50000e-03, dt = 2.50000e-03, wall = 1.64902e-01
+dt limiter: Initial timestep
+End cycle     10, time = 2.85593e-02, dt = 2.58849e-03, wall = 1.72612e+00
+dt limiter: PE 0, Hydro dV/V limit for z = 0
+
+Run complete
+cycle =     10,         cstop =     10
+time  =   2.855932e-02, tstop =   1.000000e+00
+
+************************************
+hydro cycle run time=   1.892289e+00
+************************************
+Energy check:  total energy  =   2.512181e-01
+(internal =   1.874053e-01, kinetic =   6.381282e-02)
+Writing .xy file...
+METRICS OPERATOR COLLECTION END
+```
+
+The above shows the structured output that is done in a way for our Python parsing script to easily
+find sections of data. Also note that the worker will only be alive long enough for the main job to
+finish, and once it does, the worker goes away! Here is what you'll see in its brief life:
+
+```console
+METADATA START {"pods":2,"completions":2,"metricName":"app-pennant","metricDescription":"Unstructured mesh hydrodynamics for advanced architectures ","metricType":"standalone","metricOptions":{"command":"pennant /opt/pennant/test/sedovsmall/sedovsmall.pnt","completions":0,"mpirun":"mpirun --hostfile ./hostlist.txt","rate":10,"workdir":"/opt/pennant/test"}}
+METADATA END
+Sleeping for 10 seconds waiting for network...
+METRICS OPERATOR COLLECTION START
+```
+
+We never actually parse the output of the worker, so it isn't important.
+We can do this with JobSet logic that the entire set is done when the launcher is done.
+
+```bash
+$ kubectl get pods
+```
+```console
+NAME                           READY   STATUS      RESTARTS   AGE
+metricset-sample-l-0-0-vfz4w   0/1     Completed   0          68s
+```
+
+When you are done, the job and jobset will be completed.
+
+```bash
+$ kubectl get jobset
+```
+```console
+NAME               RESTARTS   COMPLETED   AGE
+metricset-sample              True        82s
+```
+```bash
+$ kubectl get jobs
+```
+```console
+NAME                   COMPLETIONS   DURATION   AGE
+metricset-sample-n-0   1/1           18s        84s
+```
+
+And then you can cleanup!
+
+```bash
+kubectl delete -f metrics.yaml
+```

examples/tests/app-pennant/metrics.yaml

@@ -0,0 +1,13 @@
+apiVersion: flux-framework.org/v1alpha1
+kind: MetricSet
+metadata:
+  labels:
+    app.kubernetes.io/name: metricset
+    app.kubernetes.io/instance: metricset-sample
+  name: metricset-sample
+spec:
+  # Number of indexed jobs to run netmark on
+  pods: 2
+  metrics:
+   # This uses the default commands
+   - name: app-pennant