ND Tier 3 Minimal Example

sites/nd-tier3/README.md

@@ -0,0 +1,17 @@
+To run Task/Dask Vine on the ND Tier 3:
+
+```
+conda create -n taskvine python=3 -y
+conda activate taskvine
+conda install ndcctools -y
+python -m pip install awkward dask_awkward pyarrow pandas
+poncho_package_create taskvine taskvine_packed.tar.gz
+```
+
+From here, you will need two interactive sessions. One to run the application and one run the Vine Factory. A second login session will work, but using terminal multiplexer (e.g. tmux, screen) might be simpler. 
+
+You will need to edit a few things before the examples are ready to run:
+   1. In `example.py`, adjust the run argument of `run_info_path` to an area where you would like to store the files for the manager, staging files, and log files for this application.
+   2. In `factory_command.sh`, adjust the argument for `scratch_dir` to point to an appropriate scratch directory.
+
+In the first terminal, do `python example.py`. When a progress bar appears, the application is ready to receive workers. In the second terminal, do `source factory_command.sh`. When a status table appears listing the number of waiting, running, and completed tasks, the factory is ready to request and connect workers. If the application `example.py` runs successfully, you have successfully set up TaskVine.

sites/nd-tier3/example.py

@@ -0,0 +1,31 @@
+import awkward as ak
+import dask_awkward as dak
+from ndcctools.taskvine import DaskVine
+import dask
+
+# Create DaskVine
+if __name__ == "__main__":
+    m = DaskVine(
+        [9101, 9200], #Define the available ports for DaskVine here#
+        name="example_manager", #Define the name of the Vine manager here#
+        run_info_path="/path/to/vine-run-info", #Define the location for the vine-run-info directory here#,
+        run_info_template="example_application", #Define the name of the folder information for this application will be stored in here#,
+    )
+
+#Basic Options for DaksVine
+    m.tune("temp-replica-count", 3)
+    m.tune("worker-source-max-transfers", 100000)
+
+    arr = dak.from_parquet('example_input.parquet')
+    arr_sum = ak.sum(arr)
+
+    computed_sum = dask.compute(
+            arr_sum,
+            scheduler=m.get,
+            worker_transfers=True,
+            resources={"cores": 1},
+            task_mode="function-calls",
+            lib_resources={'cores': 1, 'slots': 1},
+        )
+
+    print(f'The sum is {computed_sum}; if that number is 4998.4362, then you have successfully run DaskVine!')

sites/nd-tier3/factory.json

@@ -0,0 +1,11 @@
+{
+	"manager-name":"example_manager",
+	"max-workers":2,
+	"min-workers":0,
+	"workers-per-cycle":1000,
+	"cores":2,
+	"memory":cores*1048,
+	"disk":1000,
+	"timeout":600,
+        "condor-requirements": "((has_vast) && !regexp(\"^camd\", MachineShortName))"
+}

sites/nd-tier3/factory_command.sh

@@ -0,0 +1,6 @@
+vine_factory -T condor \
+    -C factory.json \
+    --python-env taskvine_packed.tar.gz \
+    --scratch-dir /path/to/vine_scratch/ \
+    --parent-death \
+    --factory-timeout 600