Updated workflow

Author: aperez-worklytics

.github/workflows/release.yml

@@ -3,6 +3,13 @@ name: release
 on:
   push:
     branches: [ main ]
+  # Allow manual dispatch to choose publishing target (pypi or testpypi)
+  workflow_dispatch:
+    inputs:
+      publish_target:
+        description: 'Where to publish: pypi or testpypi'
+        required: false
+        default: 'pypi'
 
 jobs:
   test:
@@ -37,7 +44,9 @@ jobs:
         run: |
           pip-audit -r requirements.txt || true
       - name: Semantic Release (version, tag, GitHub release, PyPI)
+        # Choose which PYPI token to provide based on the optional dispatch input.
+        # If manually dispatched with publish_target=testpypi, the workflow will use TEST_PYPI_API_TOKEN.
         env:
-          PYPI_TOKEN: ${{ secrets.PYPI_API_TOKEN }}
+          PYPI_TOKEN: ${{ (github.event_name == 'workflow_dispatch' && github.event.inputs.publish_target == 'testpypi') && secrets.TEST_PYPI_API_TOKEN || secrets.PYPI_API_TOKEN }}
           GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         run: semantic-release publish

README.md

@@ -34,15 +34,36 @@ pip install -e .
 - Any CLI flag overrides values from `config.yaml`.
 - If neither config nor flags provide a value, the tool falls back to environment variables (for emulator detection) or sensible defaults.
 
-Example `config.yaml`:
+Example `config.yaml` (full example with comments):
 
 ```yaml
-project_id: "my-project"          # If omitted, ADC/env will be used
-emulator_host: "localhost:8010"   # If set, uses Datastore Emulator
+# Project / environment
+project_id: "my-project"          # (string) GCP project id. If omitted, ADC or DATASTORE_PROJECT_ID env var will be used.
+emulator_host: "localhost:8010"   # (string) Datastore emulator host (host:port). If set, the emulator path is used.
+
+# Explicit filters (empty -> iterate all)
+namespaces: [""]                   # (list) Namespaces to include. [""] means include default namespace and allow discovery of others.
+kinds: []                            # (list) Kinds to include. Empty/omit means discover all kinds per namespace.
+
+# Defaults used by some commands (optional)
+kind: ""                            # (string) Default kind used by analyze-fields when CLI --kind is not provided.
+namespace: ""                       # (string) Default namespace used when CLI --namespace is omitted.
+
+# Cleanup settings
+ttl_field: "expireAt"               # (string) Property name that contains the TTL/expiry timestamp.
+delete_missing_ttl: true              # (bool) If true, entities missing the TTL field will be deleted by cleanup.
+batch_size: 500                       # (int) Number of keys to delete per batch when running cleanup (tunable).
+
+# Analysis settings
+group_by_field: null                  # (string|null) Field name to group analysis by (e.g., batchId). Null means no grouping.
+sample_size: 500                      # (int) Max entities to sample per-kind/per-group to bound analysis work. Set 0 or null to disable sampling.
+enable_parallel: true                 # (bool) Enable multi-threaded processing for analysis and deletion. Set false to force single-threaded.
+
+# Logging
+log_level: "INFO"                   # (string) Logging level (DEBUG, INFO, WARNING, ERROR).
+```
 
-# Explicit filters (empty means all)
-namespaces: [""]                   # Empty -> iterate all namespaces (including default "")
-kinds: []                          # Empty -> iterate all kinds per namespace
+The keys above map directly to CLI flags (CLI flags override values in `config.yaml`). Omit any option to use sensible defaults.
 
  # local-storage-utils — Quickstart
 
@@ -105,6 +126,19 @@ Development & tests
 - Run full test suite locally:
   - `make integration`
 
+Publishing
+-------
+
+This project uses the `release` workflow to publish releases to PyPI. Follow the packaging tutorial for a complete guide on packaging and publishing: https://packaging.python.org/en/latest/tutorials/packaging-projects/
+
+We support publishing to either TestPyPI (for dry runs) or the real PyPI. The workflow can be triggered automatically on pushes to `main` or manually via the Actions UI (use the "Run workflow" button). When you run it manually you can set the `publish_target` input to `testpypi` to publish to TestPyPI instead of PyPI.
+
+Secrets and tokens
+- For production publishing to the real PyPI, set the repository secret named `PYPI_API_TOKEN` with a PyPI API token.
+- For test publishing to TestPyPI, set the repository secret named `TEST_PYPI_API_TOKEN` with a TestPyPI API token.
+
+The release workflow selects the appropriate token based on the `publish_target` input. Use TestPyPI first to validate packaging and metadata before publishing to the real index.
+
 Notes
 - `sample_size` bounds per-kind/group analysis to avoid scanning entire datasets. Set to 0 or `null` in config to disable sampling.
 - `enable_parallel` (default true) enables multi-threaded processing during analysis and deletion; set to false to force single-threaded behavior.