Skip to content

feat: GitHub Actions - Validate Remote Manifest Against Local Schema Before Running Validate Modified Targets Test Suite #2618

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

feat: GitHub Actions - Validate Remote Manifest Against Local Schema Before Running Validate Modified Targets Test Suite #2618

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7cbc4a5
Validate head manifest against base schema
MaxwellOldshein Oct 6, 2025
8d9f426
Fixed imports for schema validation exceptions
MaxwellOldshein Oct 6, 2025
26d132c
Update validation error messages
MaxwellOldshein Oct 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .github/workflows/validate_modified_targets.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ jobs:
# Safely extract the file from the PR's head and the merge-base commit
git show pr:sherlock_project/resources/data.json > data.json.head
git show $MERGE_BASE:sherlock_project/resources/data.json > data.json.base
git show $MERGE_BASE:sherlock_project/resources/data.schema.json > data.schema.json.base

# CRITICAL FIX: Overwrite the checked-out data.json with the one from the PR
# This ensures that pytest runs against the new, updated file.
Expand All @@ -59,15 +60,25 @@ jobs:
python - <<'EOF'
import json
import sys
from jsonschema import validate, SchemaError, ValidationError
try:
with open("data.json.base") as f: base = json.load(f)
with open("data.schema.json.base") as f: schema = json.load(f)
with open("data.json.head") as f: head = json.load(f)

validate(instance=head, schema=schema)
except FileNotFoundError as e:
print(f"Error: Could not find {e.filename}", file=sys.stderr)
sys.exit(1)
except json.JSONDecodeError as e:
print(f"Error: Could not decode JSON from a file - {e}", file=sys.stderr)
sys.exit(1)
except jsonschema.exceptions.SchemaError as e:
print(f"Error: Could not locate local JSON schema for validation - {e}", file=sys.stderr)
sys.exit(1)
except jsonschema.exceptions.ValidationError as e:
print(f"Error: Could not validate remote manifest against local schema - {e}", file=sys.stderr)
sys.exit(1)

changed = []
for k, v in head.items():
Expand Down