Add atlasopenmagic DID finder

Author: ponyisi

.github/workflows/deploy-config.json

@@ -14,6 +14,11 @@
     "image_name": "servicex-did-finder-cernopendata",
     "test_required": true
   },
+  {
+    "dir_name": "did_finder_atlasopenmagic",
+    "image_name": "servicex-did-finder-atlasopenmagic",
+    "test_required": true
+  },
   {
     "dir_name": "did_finder_rucio",
     "image_name": "servicex-did-finder",

did_finder_atlasopenmagic/.coveragerc

@@ -0,0 +1,13 @@
+[run]
+branch = True
+source = src/
+
+[report]
+exclude_lines =
+    if self.debug:
+    pragma: no cover
+    raise NotImplementedError
+    if __name__ == .__main__.:
+ignore_errors = True
+omit =
+    tests/*

did_finder_atlasopenmagic/.flake8

@@ -0,0 +1,2 @@
+[flake8]
+max-line-length=99

did_finder_atlasopenmagic/.gitattributes

@@ -0,0 +1,2 @@
+# Auto detect text files and perform LF normalization
+* text=auto

did_finder_atlasopenmagic/.gitignore

@@ -0,0 +1,117 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+.python-version
+
+# celery beat schedule file
+celerybeat-schedule
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+.vscode/settings.json
+
+.idea/

did_finder_atlasopenmagic/Dockerfile

@@ -0,0 +1,41 @@
+FROM python:3.14
+
+LABEL maintainer="Peter Onyisi <[email protected]>"
+
+USER root
+
+# Create app directory
+WORKDIR /opt/servicex
+
+# Create celery user. Assign them to group zero since that is the group OpenShift will run the container as
+RUN useradd -g 0 -ms /bin/bash celery
+
+ENV  POETRY_VERSION=2.3
+
+RUN pip install poetry==$POETRY_VERSION
+
+COPY pyproject.toml pyproject.toml
+COPY poetry.lock poetry.lock
+
+# Bring over the main scripts
+COPY . .
+
+ENV XDG_CONFIG_HOME=/opt/servicex
+RUN poetry config virtualenvs.in-project true
+RUN poetry install --no-root --no-interaction --no-ansi
+
+# Change ownership of the app directory to celery user. Also set the group to zero since
+# that is the group OpenShift will run the container as
+RUN chown -R celery:0 /opt/servicex
+RUN chmod -R g=u /opt/servicex
+
+# Switch to celery user
+USER celery
+
+# Make sure python isn't buffered
+ENV PYTHONUNBUFFERED=1
+
+ENV PYTHONPATH=/opt/servicex/src
+ENV BROKER_URL="amqp://guest:guest@localhost:5672//"
+
+ENTRYPOINT [ "scripts/start_celery_worker.sh"]

did_finder_atlasopenmagic/README.md

@@ -0,0 +1,10 @@
+# ServiceX_DID_finder_ATLASOpenMagic
+ Access datasets for ServiceX with `atlasopenmagic`.
+
+## Finding datasets
+
+The ATLAS experiment hosts releases of [ATLAS Open Data](https://opendata.atlas.cern/). The proton-proton simulations and data are ROOT files, which come in two flavors: [education](https://opendata.atlas.cern/docs/category/data-for-education) and [research](https://opendata.atlas.cern/docs/category/data-for-research). The former are flat ROOT files with a limited set of branches, and the latter are in the PHYSLITE format which is also readable via `uproot` although with a more complicated schema (the `coffea` PHYSLITE schema is recommended for analysis). There are additional datasets for heavy ion collisions (also ROOT) and pure event generator output (in HEPMC format).
+
+The [`atlasopenmagic`](https://github.com/atlas-outreach-data-tools/atlasopenmagic) package is provided as an interface to this data. To look up the files for a dataset, you must provide the _release_ for the dataset, as well as the _dataset ID_ (this is either a numeric string, corresponding to the ATLAS Monte Carlo simulation sample ID, or the string "data" for data). Optionally, where supported by the release, a _skim_ can be specifed. The `atlasopenmagic` DID finder accepts a single string encoding all these, which must be of the form `<release>/<dataset_id>` or `<release>/<dataset_id>/<skim>` where the appropriate replacements are made. The assumption is that the user will use `atlasopenmagic` directly in their code to handle the metadata aspects of using the ATLAS Open Data (e.g.\ sample cross sections, initial number of events, etc.) while using ServiceX to apply event selection and column reduction to the files.
+
+Check the `helm/servicex/templates/did-finder-atlasopenmagic/deployment.yaml` file for an example of how to deploy this DID finder.