feat(tests): add functional test suite for real API testing#13
Merged
feat(tests): add functional test suite for real API testing#13
Conversation
Add opt-in functional tests that exercise HawkOp commands against the real StackHawk API with safety guards for production environments. Test infrastructure: - Add `functional-tests` Cargo feature (opt-in, not in default) - Add FunctionalTestContext with production API safety checks - Add TestTeam RAII wrapper for automatic cleanup - Add predicates dev dependency for flexible assertions Test coverage (75 tests total): - 34 read-only tests (org, app, scan, user, team, policy, repo, oas, config, secret, audit, cache, version, help, completion commands) - 18 mutation tests with automatic cleanup using hawkop-functest-* naming - 18 error scenario tests for invalid IDs, missing args, etc. - 2 tests ignored (require hosted scan configs feature) Safety features: - Production API warning banner with HAWKOP_FUNCTIONAL_TESTS_CONFIRM=yes - Profile-based configuration via HAWKOP_PROFILE - Dry-run preview mode (lists tests without execution) Makefile targets: - `make functional-test` - run tests (requires HAWKOP_PROFILE) - `make functional-test-dry-run` - preview test list Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Previously, when an endpoint returned 401 due to feature/role restrictions (not expired token), the client would infinitely retry after refreshing the token. This happened because there was no tracking of whether a token refresh had already been attempted. Changes: - Add `token_refreshed: bool` parameter to all retry functions to track if a token refresh has already been attempted - Add `ApiError::UnauthorizedFeature` variant for 401s after token refresh - Add `ApiError::unauthorized_feature()` helper to format helpful messages - Add `get_current_user_role()` method to fetch user's org role for errors - Error message now shows: endpoint URL, user's role, and possible causes Affected functions: - request_with_retry() - request_with_body_retry() - request_delete_with_retry() Example error output: Access denied. The endpoint 'https://api.stackhawk.com/api/v1/configuration/...' may require a feature flag not enabled for your organization, or elevated privileges. Your current role: MEMBER Possible causes: • This feature requires a plan upgrade or feature flag • Your role may not have access (Owner/Admin/Member) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…vior - Fix dry-run validation order in team commands (add_user, remove_user, add_app, remove_app) - now checks --dry-run before resolving users/apps - Add bad_request_message() helper for contextual 400 error messages - Add run_feature_flag_dependent() test helper for graceful handling of missing feature flags (oas, config endpoints) - Fix test_cache_path_shows_path to check for directory not filename - Remove invalid test_status_json_format (status doesn't support --format) - Remove invalid test_scan_get_missing_id (scan get shows latest by design) - Add --nocapture to Makefile so feature flag warnings are visible Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Key Changes
Functional Test Suite (73 tests)
HAWKOP_FUNCTIONAL_TESTS_CONFIRM=yesBug Fixes
--dry-runbefore validating user/app existence, allowing previews with placeholder IDshawkop org listto see available organizations."Test Plan
HAWKOP_PROFILE=test make functional-testcargo testUsage
🤖 Generated with Claude Code