Commit 6fc38bb
feat: add entity relationships API support with optimized backup/restore (#160)
* chore: update HISTORY.md for main
* perf: optimize test scheduling with --dist loadfile for 25% faster test runs (#157)
* refactor: separate trigger-evaluation test to avoid scorecard evaluation race conditions
- Create dedicated cli-test-evaluation-scorecard for trigger-evaluation testing
- Remove retry logic complexity from test_scorecards() and test_scorecards_drafts()
- Add new test_scorecard_trigger_evaluation() that creates/deletes its own scorecard
- Eliminates race condition where import triggers evaluation conflicting with tests
* refactor: remove unnecessary mock decorator from _get_rule helper function
The helper function doesn't need its own environment patching since it's called
from fixtures that already have their own @mock.patch.dict decorators.
* Revert "perf: optimize test scheduling with --dist loadfile for 25% faster test runs (#157)"
This reverts commit 8879fcf.
The --dist loadfile optimization caused race conditions between tests that
share resources (e.g., test_custom_events_uuid and test_custom_events_list
both operate on custom events and can interfere with each other when run in
parallel by file).
Reliability > speed. Better to have tests take 40s with no race conditions
than 30s with intermittent failures.
* perf: rename test_deploys.py to test_000_deploys.py for early scheduling
Pytest collects tests alphabetically by filename. With pytest-xdist --dist load,
tests collected earlier are more likely to be scheduled first. Since test_deploys
is the longest-running test (~19s), scheduling it early maximizes parallel
efficiency with 12 workers.
This is our general strategy: prefix slow tests with numbers (000, 001, etc.) to
control scheduling order without introducing race conditions like --dist loadfile.
* feat: add entity relationships API support and fix backup export bug
- Fix double-encoding bug in backup export for entity-types and ip-allowlist
- entity-types and ip-allowlist were being converted to strings before json.dump
- This caused import failures with "TypeError: string indices must be integers"
- Add entity-relationship-types commands:
- list: List all relationship types
- get: Get relationship type by tag
- create: Create new relationship type
- update: Update existing relationship type
- delete: Delete relationship type
- Add entity-relationships commands:
- list: List all relationships for a type
- list-destinations: Get destinations for source entity
- list-sources: Get sources for destination entity
- add-destinations: Add destinations to source
- add-sources: Add sources to destination
- update-destinations: Replace all destinations for source
- update-sources: Replace all sources for destination
- add-bulk: Add multiple relationships
- update-bulk: Replace all relationships for type
- Integrate entity relationships into backup/restore:
- Export entity-relationship-types and entity-relationships
- Import with proper ordering (types before catalog, relationships after)
- Transform export format to bulk update format for import
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <[email protected]>
* fix: clean up entity relationships import output and fix bugs
- Add _print parameter to entity_relationship_types.create() and entity_relationships.update_bulk()
- Use _print=False when importing to suppress JSON output
- Fix import to use correct keys: sourceEntity.tag and destinationEntity.tag instead of source.tag
- Replace typer.unstable.TempFile with standard tempfile.NamedTemporaryFile
- Improve output: show only tag names instead of full JSON when importing
- Add missing tempfile import
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <[email protected]>
* fix: support re-importing existing entity relationship types
- Check if relationship type already exists before importing
- Use update instead of create for existing relationship types
- Add _print parameter to entity_relationship_types.update()
- Matches pattern used by entity_types import
This allows backup imports to be idempotent and run multiple times
without encountering "already exists" errors.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <[email protected]>
* feat: improve error handling in backup import
- Add detailed error reporting for catalog imports
- Show filename, error type, and error message for failures
- Add total failure count at end of catalog import
- Add error handling for entity relationship type imports
- Wrap create/update in try/except blocks
- Show which relationship type failed and why
- Add total failure count
- Add error handling for entity relationship imports
- Wrap import operations in try/except blocks
- Show which relationship type failed and why
- Add total failure count
This makes it much easier to diagnose import failures by showing
exactly which files are failing and what the error is.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <[email protected]>
* fix: improve catalog import error handling and make sequential
- Change catalog import from parallel to sequential execution
- This allows errors to be correlated with specific files
- HTTP errors from cortex_client are now shown with filenames
- Catch typer.Exit exceptions in catalog import
- The HTTP client raises typer.Exit on errors
- Now catches and reports which file caused the error
- Remove unused imports added for parallel error capture
- Simplify catalog import logic
Note: The plugin import failures with "string indices must be integers"
are due to exports created before the double-encoding bug fix. Re-export
with the current code to fix these.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <[email protected]>
* perf: parallelize entity relationships and catalog imports
- Restore parallel execution for catalog import (30 workers)
- Previously made sequential for error debugging
- Now handles typer.Exit exceptions properly
- Maintains good error reporting with filenames
- Parallelize entity relationship type imports (30 workers)
- Check existing types once, then import in parallel
- Properly handles create vs update decision
- Parallelize entity relationship imports (30 workers)
- Each relationship type is independent
- Can safely import in parallel
All imports now use ThreadPoolExecutor with 30 workers for
maximum performance while maintaining error reporting.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <[email protected]>
* feat: add comprehensive import summary and retry commands
- All import functions now return statistics (type, imported count, failures)
- Import summary section shows:
- Per-type import counts and failures
- Total imported and failed counts
- Failed imports section lists:
- Full file paths for all failures
- Error type and message for each
- Retry commands section provides:
- Ready-to-run cortex commands for each failed file
- Can copy/paste directly from terminal
- Commands use proper quoting for file paths with spaces
- Updated all import functions to track file paths in parallel execution
- Added typer.Exit exception handling to plugins, scorecards, workflows
- Consistent error reporting across all import types
Example output:
IMPORT SUMMARY
catalog: 250 imported, 5 failed
TOTAL: 500 imported, 5 failed
FAILED IMPORTS
/path/to/file.yaml
Error: HTTP - Validation or HTTP error
RETRY COMMANDS
cortex catalog create -f "/path/to/file.yaml"
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <[email protected]>
* fix: show catalog filename before import attempt
- Print "Importing: filename" before attempting catalog import
- HTTP errors now appear immediately after the filename
- Remove duplicate success messages at end
- Only show failure count summary
This makes it immediately clear which file is causing each HTTP 400 error:
Before:
Processing: .../catalog
HTTP Error 400: Unknown error
HTTP Error 400: Unknown error
After:
Processing: .../catalog
Importing: docs.yaml
HTTP Error 400: Unknown error
Importing: another.yaml
HTTP Error 400: Unknown error
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <[email protected]>
* fix: improve test isolation for custom events list test
Delete all event types (not just VALIDATE_SERVICE) at test start to prevent
interference from parallel tests. The connection pooling performance improvements
made tests run much faster, increasing temporal overlap between parallel tests
and exposing this existing test isolation issue.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <[email protected]>
* fix: initialize variables before conditional to prevent NameError
When entity-relationship-types or entity-relationships directories don't exist
(like in test data), the import functions would reference undefined `results` and
`failed_count` variables, causing a NameError and preventing subsequent imports
from running (including catalog import, breaking tests).
This bug was causing test_catalog_delete_entity and test_custom_events_list to
fail because the import would crash before importing catalog entities.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <[email protected]>
---------
Co-authored-by: GitHub Actions <[email protected]>
Co-authored-by: Claude <[email protected]>1 parent 32c9f45 commit 6fc38bb
File tree
5 files changed
+644
-39
lines changed- cortexapps_cli
- commands
- tests
5 files changed
+644
-39
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
| 27 | + | |
| 28 | + | |
27 | 29 | | |
28 | 30 | | |
29 | 31 | | |
| |||
59 | 61 | | |
60 | 62 | | |
61 | 63 | | |
| 64 | + | |
| 65 | + | |
62 | 66 | | |
63 | 67 | | |
64 | 68 | | |
| |||
0 commit comments