Skip to content

refactor(ts)!: use @actions/exec instead of bash script for OPA commands, allow testing entire directory vs file by file#27

Merged
oycyc merged 56 commits intomainfrom
refactor/use-json-output
May 27, 2025
Merged

refactor(ts)!: use @actions/exec instead of bash script for OPA commands, allow testing entire directory vs file by file#27
oycyc merged 56 commits intomainfrom
refactor/use-json-output

Conversation

@oycyc
Copy link
Collaborator

@oycyc oycyc commented May 12, 2025

Describe your changes

  • Input var of test_mode to indicate whether to test the Rego by an entire directory - directory - (including entire package, e.g. opa test ./) or by individual file by file basis - file - (e.g. opa test a_test.rego a.rego). Default will test by entire directory package.
    • Previously, it was testing OPA policies on a file by file basis (testing a.rego against a_test.rego). If this is the desired behavior, use test_mode = file.
  • Default OPA version is now the latest at the time of writing at 1.4.2. Previously, it was 0.67.1, latest at Summer of 2024.
  • The action will now use @actions/exec to run the OPA test commands instead of a bash script. The output is consumed and processed as JSON.

Reminder:

  • When the PR is ready, be sure to run npm run build to compile into the distribution /dist folder, which is the source code that the Action uses.

Summary by CodeRabbit

  • New Features

    • Added support for running OPA tests in both directory and individual file modes, with improved configuration options for test execution and coverage reporting.
    • Introduced Markdown-formatted test and coverage summaries for enhanced GitHub PR comments.
    • Added Visual Studio Code debug configuration for easier local development.
  • Bug Fixes

    • Improved handling and reporting of test and coverage results, including more accurate file path and line coverage reporting.
  • Documentation

    • Updated usage instructions and input descriptions in the README, including new configuration options and examples.
    • Added documentation for example OPA policies.
  • Refactor

    • Modularized test execution, result processing, and formatting logic for better maintainability and clarity.
  • Tests

    • Updated and expanded test suites and mock data to reflect new processing logic and output formats.

@coderabbitai
Copy link

coderabbitai bot commented May 12, 2025

Caution

Review failed

The pull request is closed.

Walkthrough

The changes introduce a modular refactor of the OPA Rego test GitHub Action, shifting from inline parsing and execution to structured, type-safe modules for test and coverage processing. New configuration options, improved documentation, and updated workflows support both directory and file-based test modes. Test output formats and coverage reporting are modernized and standardized.

Changes

File(s) / Group Change Summary
.github/workflows/test.yml Split test workflow into individual file and directory package modes; updated PR comment titles and inputs.
.vscode/launch.json Added VS Code launch config for debugging with ts-node and environment variables for local testing.
README.md, examples/README.md Updated documentation for new inputs, usage, and debugging; added license note for examples.
action.yml Made path input required, added test_mode, updated defaults and descriptions, removed composite test step, improved PR comment tagging and action versioning.
package.json Added scripts for local/dev runs, added @actions/exec dependency, improved Jest config.
src/interfaces.ts Introduced TypeScript interfaces for processed test and coverage results, and raw OPA outputs.
src/opaCommands.ts New module to execute OPA tests/coverage by directory or file, capturing outputs and errors.
src/testResultProcessing.ts New module to process raw OPA test and coverage JSON into summarized, display-ready results.
src/formatResults.ts New function to format processed results into a Markdown table for PR comments, with optional coverage.
src/index.ts Refactored to use modular processing and execution; removed inline parsing/formatting; updated control flow for new test modes.
__tests__/index.test.ts, __tests__/mockResults.ts Refactored tests to use new processing/formatting modules and mock data; updated expectations for new formats.
__tests__/sample_coverage_output.txt, __tests__/sample_test_output.txt Updated test and coverage output files to new JSON formats and coverage stats.
examples/ignore-changes-outside-root.rego Updated policy rules to use explicit if conditions and added import statement.
examples/tests/enforce-module-use-policy_test.rego Removed one test case for controlled resource deletion.
__tests__/sample_test_success_fail_mixed.txt Deleted obsolete test output file with old format.

Sequence Diagram(s)

sequenceDiagram
    participant GH_Action as GitHub Action
    participant OpaCmd as opaCommands.ts
    participant TestProc as testResultProcessing.ts
    participant Format as formatResults.ts

    GH_Action->>OpaCmd: Execute OPA tests (by directory or file)
    OpaCmd-->>GH_Action: Return raw JSON test & coverage output
    GH_Action->>TestProc: Process raw test results
    GH_Action->>TestProc: Process raw coverage report
    TestProc-->>GH_Action: Return processed results
    GH_Action->>Format: Format results for PR comment
    Format-->>GH_Action: Return Markdown table
    GH_Action->>GH_Action: Post comment to PR
Loading

Possibly related PRs

Poem

In the warren of code, a rabbit hopped,
Refactoring modules, old steps were dropped.
Now tests run by file, or the whole directory,
With coverage and outputs in JSON clarity.
Markdown tables in PRs, so neat and bright,
This bunny’s action now works just right! 🐇✨


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 6d75705 and 6852b79.

⛔ Files ignored due to path filters (3)
  • dist/index.js is excluded by !**/dist/**
  • dist/index.js.map is excluded by !**/dist/**, !**/*.map
  • package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (18)
  • .github/workflows/test.yml (1 hunks)
  • .vscode/launch.json (1 hunks)
  • README.md (4 hunks)
  • __tests__/index.test.ts (8 hunks)
  • __tests__/mockResults.ts (1 hunks)
  • __tests__/sample_coverage_output.txt (3 hunks)
  • __tests__/sample_test_output.txt (1 hunks)
  • __tests__/sample_test_success_fail_mixed.txt (0 hunks)
  • action.yml (3 hunks)
  • examples/README.md (1 hunks)
  • examples/ignore-changes-outside-root.rego (2 hunks)
  • examples/tests/enforce-module-use-policy_test.rego (1 hunks)
  • package.json (3 hunks)
  • src/formatResults.ts (1 hunks)
  • src/index.ts (2 hunks)
  • src/interfaces.ts (1 hunks)
  • src/opaCommands.ts (1 hunks)
  • src/testResultProcessing.ts (1 hunks)
✨ Finishing Touches
  • 📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@masterpointio masterpointio deleted a comment from github-actions bot May 26, 2025
@github-actions
Copy link
Contributor

github-actions bot commented May 26, 2025

Coverage report

St.
Category Percentage Covered / Total
🟢 Statements
95.56% (+5.49% 🔼)
86/90
🟢 Branches
92.59% (+8.09% 🔼)
25/27
🟢 Functions
100% (+18.18% 🔼)
8/8
🟢 Lines
96.63% (+6.22% 🔼)
86/89
Show new covered files 🐣
St.
File Statements Branches Functions Lines
🟢
... / mockResults.ts
100% 100% 100% 100%
🟢
... / testResultProcessing.ts
100% 100% 100% 100%
🟢
... / formatResults.ts
92% 89.47% 100% 93.88%

Test suite run success

15 tests passing in 1 suite.

Report generated by 🧪jest coverage report action from 6852b79

@masterpointio masterpointio deleted a comment from github-actions bot May 26, 2025
@masterpointio masterpointio deleted a comment from github-actions bot May 26, 2025
@masterpointio masterpointio deleted a comment from github-actions bot May 26, 2025
@oycyc oycyc requested a review from Copilot May 27, 2025 10:30
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR enables the Action to test itself by running against its own policy files and examples, with changes across rego policies, tests, workflow configurations, and project metadata.

  • Introduces new rego policies and tests for push, cancel and file validations.
  • Updates package metadata and workflow files to support both individual file and directory package testing.
  • Adjusts test output processing and result formatting in the testing framework.

Reviewed Changes

Copilot reviewed 34 out of 34 changed files in this pull request and generated no comments.

Show a summary per file
File Description
spacelift_policies/**/*.rego New and updated rego policies/tests to validate push‐policy behavior and cancellation logic.
package.json Updated scripts and dependencies to support local testing and processing of test results.
action.yml Revised input definitions and documentation to clarify test mode and result reporting.
.github/workflows/test.yml Workflow updated with separate job configurations for file and package testing.
tests/* Test files updated to parse and format new test/coverage output formats.
.vscode/launch.json Debug configuration updated to support different test modes.
Comments suppressed due to low confidence (3)

examples/tests/enforce-module-use-policy_test.rego:19

  • [nitpick] A test case was removed and replaced with a comment '# TO ADD BACK'. Please clarify if this removal is intentional or if the test case should be reintroduced to avoid confusion during future maintenance.
-test_allow_deletion_of_controlled_resource_type {

action.yml:25

  • The input default for write_pr_comment (and similar inputs) has been changed to a string value. Ensure that this change aligns with the expected input types across the Action to avoid type inconsistencies.
default: "true"

spacelift_policies/push_package/cancel_test.rego:29

  • The variable 'main_stack' is referenced but not defined in this file. Consider importing or declaring 'main_stack' to ensure the test runs as intended.
"base": {"branch": main_stack.branch},

@oycyc oycyc changed the title test action on itself refactor: use @actions/exec instead of bash script for OPA commands, allow testing entire directory vs file by file May 27, 2025
@oycyc oycyc changed the title refactor: use @actions/exec instead of bash script for OPA commands, allow testing entire directory vs file by file refactor(ts): use @actions/exec instead of bash script for OPA commands, allow testing entire directory vs file by file May 27, 2025
@oycyc oycyc changed the title refactor(ts): use @actions/exec instead of bash script for OPA commands, allow testing entire directory vs file by file refactor(ts)!: use @actions/exec instead of bash script for OPA commands, allow testing entire directory vs file by file May 27, 2025
@oycyc oycyc marked this pull request as ready for review May 27, 2025 13:41
@oycyc oycyc merged commit 9eecaf7 into main May 27, 2025
7 of 8 checks passed
@oycyc oycyc deleted the refactor/use-json-output branch May 27, 2025 13:42
oycyc pushed a commit that referenced this pull request May 30, 2025
🤖 I have created a release *beep* *boop*
---


##
[2.0.0](v1.2.0...v2.0.0)
(2025-05-30)

This new release changes how the OPA test commands are executed.
Previously, it was executed as a bash script within the `action.yml`.
Now, it is executed programmatically with [GitHub's `@action/exec`
package
](https://github.com/actions/toolkit/tree/main/packages/exec). This
allows the output to be easily fed as JSON and processed.

There is also a new input option of `test_mode` which allows the option
to test the OPA by the entire package (directory based aka `opa test
./`) or file by file (e.g. `opa test a_test.rego a.rego`).
* The end result should be the same if the OPA package is structured
properly.
* Previously, it was tested file by file, so if tests are failing, to
maintain old behavior, use the input `test_mode = file`
* If encountering any issues, please be sure to report them!


### ⚠ BREAKING CHANGES

* **ts:** use @actions/exec instead of bash script for OPA commands,
allow testing entire directory vs file by file
([#27](#27))

### Bug Fixes

* **ts:** coverage only if it passed
([#30](#30))
([7e4e69c](7e4e69c))


### Code Refactoring

* **ts:** use @actions/exec instead of bash script for OPA commands,
allow testing entire directory vs file by file
([#27](#27))
([9eecaf7](9eecaf7))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant