Skip to content

Commit 6fc38bb

Browse files
jeff-schnitteractions-userclaude
authored
feat: add entity relationships API support with optimized backup/restore (#160)
* chore: update HISTORY.md for main * perf: optimize test scheduling with --dist loadfile for 25% faster test runs (#157) * refactor: separate trigger-evaluation test to avoid scorecard evaluation race conditions - Create dedicated cli-test-evaluation-scorecard for trigger-evaluation testing - Remove retry logic complexity from test_scorecards() and test_scorecards_drafts() - Add new test_scorecard_trigger_evaluation() that creates/deletes its own scorecard - Eliminates race condition where import triggers evaluation conflicting with tests * refactor: remove unnecessary mock decorator from _get_rule helper function The helper function doesn't need its own environment patching since it's called from fixtures that already have their own @mock.patch.dict decorators. * Revert "perf: optimize test scheduling with --dist loadfile for 25% faster test runs (#157)" This reverts commit 8879fcf. The --dist loadfile optimization caused race conditions between tests that share resources (e.g., test_custom_events_uuid and test_custom_events_list both operate on custom events and can interfere with each other when run in parallel by file). Reliability > speed. Better to have tests take 40s with no race conditions than 30s with intermittent failures. * perf: rename test_deploys.py to test_000_deploys.py for early scheduling Pytest collects tests alphabetically by filename. With pytest-xdist --dist load, tests collected earlier are more likely to be scheduled first. Since test_deploys is the longest-running test (~19s), scheduling it early maximizes parallel efficiency with 12 workers. This is our general strategy: prefix slow tests with numbers (000, 001, etc.) to control scheduling order without introducing race conditions like --dist loadfile. * feat: add entity relationships API support and fix backup export bug - Fix double-encoding bug in backup export for entity-types and ip-allowlist - entity-types and ip-allowlist were being converted to strings before json.dump - This caused import failures with "TypeError: string indices must be integers" - Add entity-relationship-types commands: - list: List all relationship types - get: Get relationship type by tag - create: Create new relationship type - update: Update existing relationship type - delete: Delete relationship type - Add entity-relationships commands: - list: List all relationships for a type - list-destinations: Get destinations for source entity - list-sources: Get sources for destination entity - add-destinations: Add destinations to source - add-sources: Add sources to destination - update-destinations: Replace all destinations for source - update-sources: Replace all sources for destination - add-bulk: Add multiple relationships - update-bulk: Replace all relationships for type - Integrate entity relationships into backup/restore: - Export entity-relationship-types and entity-relationships - Import with proper ordering (types before catalog, relationships after) - Transform export format to bulk update format for import 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * fix: clean up entity relationships import output and fix bugs - Add _print parameter to entity_relationship_types.create() and entity_relationships.update_bulk() - Use _print=False when importing to suppress JSON output - Fix import to use correct keys: sourceEntity.tag and destinationEntity.tag instead of source.tag - Replace typer.unstable.TempFile with standard tempfile.NamedTemporaryFile - Improve output: show only tag names instead of full JSON when importing - Add missing tempfile import 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * fix: support re-importing existing entity relationship types - Check if relationship type already exists before importing - Use update instead of create for existing relationship types - Add _print parameter to entity_relationship_types.update() - Matches pattern used by entity_types import This allows backup imports to be idempotent and run multiple times without encountering "already exists" errors. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * feat: improve error handling in backup import - Add detailed error reporting for catalog imports - Show filename, error type, and error message for failures - Add total failure count at end of catalog import - Add error handling for entity relationship type imports - Wrap create/update in try/except blocks - Show which relationship type failed and why - Add total failure count - Add error handling for entity relationship imports - Wrap import operations in try/except blocks - Show which relationship type failed and why - Add total failure count This makes it much easier to diagnose import failures by showing exactly which files are failing and what the error is. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * fix: improve catalog import error handling and make sequential - Change catalog import from parallel to sequential execution - This allows errors to be correlated with specific files - HTTP errors from cortex_client are now shown with filenames - Catch typer.Exit exceptions in catalog import - The HTTP client raises typer.Exit on errors - Now catches and reports which file caused the error - Remove unused imports added for parallel error capture - Simplify catalog import logic Note: The plugin import failures with "string indices must be integers" are due to exports created before the double-encoding bug fix. Re-export with the current code to fix these. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * perf: parallelize entity relationships and catalog imports - Restore parallel execution for catalog import (30 workers) - Previously made sequential for error debugging - Now handles typer.Exit exceptions properly - Maintains good error reporting with filenames - Parallelize entity relationship type imports (30 workers) - Check existing types once, then import in parallel - Properly handles create vs update decision - Parallelize entity relationship imports (30 workers) - Each relationship type is independent - Can safely import in parallel All imports now use ThreadPoolExecutor with 30 workers for maximum performance while maintaining error reporting. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * feat: add comprehensive import summary and retry commands - All import functions now return statistics (type, imported count, failures) - Import summary section shows: - Per-type import counts and failures - Total imported and failed counts - Failed imports section lists: - Full file paths for all failures - Error type and message for each - Retry commands section provides: - Ready-to-run cortex commands for each failed file - Can copy/paste directly from terminal - Commands use proper quoting for file paths with spaces - Updated all import functions to track file paths in parallel execution - Added typer.Exit exception handling to plugins, scorecards, workflows - Consistent error reporting across all import types Example output: IMPORT SUMMARY catalog: 250 imported, 5 failed TOTAL: 500 imported, 5 failed FAILED IMPORTS /path/to/file.yaml Error: HTTP - Validation or HTTP error RETRY COMMANDS cortex catalog create -f "/path/to/file.yaml" 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * fix: show catalog filename before import attempt - Print "Importing: filename" before attempting catalog import - HTTP errors now appear immediately after the filename - Remove duplicate success messages at end - Only show failure count summary This makes it immediately clear which file is causing each HTTP 400 error: Before: Processing: .../catalog HTTP Error 400: Unknown error HTTP Error 400: Unknown error After: Processing: .../catalog Importing: docs.yaml HTTP Error 400: Unknown error Importing: another.yaml HTTP Error 400: Unknown error 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * fix: improve test isolation for custom events list test Delete all event types (not just VALIDATE_SERVICE) at test start to prevent interference from parallel tests. The connection pooling performance improvements made tests run much faster, increasing temporal overlap between parallel tests and exposing this existing test isolation issue. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * fix: initialize variables before conditional to prevent NameError When entity-relationship-types or entity-relationships directories don't exist (like in test data), the import functions would reference undefined `results` and `failed_count` variables, causing a NameError and preventing subsequent imports from running (including catalog import, breaking tests). This bug was causing test_catalog_delete_entity and test_custom_events_list to fail because the import would crash before importing catalog entities. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> --------- Co-authored-by: GitHub Actions <[email protected]> Co-authored-by: Claude <[email protected]>
1 parent 32c9f45 commit 6fc38bb

File tree

5 files changed

+644
-39
lines changed

5 files changed

+644
-39
lines changed

cortexapps_cli/cli.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,8 @@
2424
import cortexapps_cli.commands.discovery_audit as discovery_audit
2525
import cortexapps_cli.commands.docs as docs
2626
import cortexapps_cli.commands.entity_types as entity_types
27+
import cortexapps_cli.commands.entity_relationship_types as entity_relationship_types
28+
import cortexapps_cli.commands.entity_relationships as entity_relationships
2729
import cortexapps_cli.commands.gitops_logs as gitops_logs
2830
import cortexapps_cli.commands.groups as groups
2931
import cortexapps_cli.commands.initiatives as initiatives
@@ -59,6 +61,8 @@
5961
app.add_typer(discovery_audit.app, name="discovery-audit")
6062
app.add_typer(docs.app, name="docs")
6163
app.add_typer(entity_types.app, name="entity-types")
64+
app.add_typer(entity_relationship_types.app, name="entity-relationship-types")
65+
app.add_typer(entity_relationships.app, name="entity-relationships")
6266
app.add_typer(gitops_logs.app, name="gitops-logs")
6367
app.add_typer(groups.app, name="groups")
6468
app.add_typer(initiatives.app, name="initiatives")

0 commit comments

Comments
 (0)