Skip to content

feat: add JSON Schema for runbook YAML format#1398

Open
k1LoW wants to merge 5 commits intomainfrom
json-schema
Open

feat: add JSON Schema for runbook YAML format#1398
k1LoW wants to merge 5 commits intomainfrom
json-schema

Conversation

@k1LoW
Copy link
Owner

@k1LoW k1LoW commented Feb 18, 2026

Add runbook.schema.json (Draft 2020-12) covering all top-level properties, runner configs, step definitions, and polymorphic types. Boolean/integer fields accept strings to support env var expansion.

Add schema_test.go with four validation tests:

  • fixture validation against all testdata/book/*.yml
  • reserved key coverage check against code constants
  • runner config field coverage via reflect
  • invalid runbook rejection

Remove unused useAgent field from sshd_local_forward_with_openapi3.yml as it has no corresponding implementation in sshRunnerConfig.

Add runbook.schema.json (Draft 2020-12) covering all top-level
properties, runner configs, step definitions, and polymorphic types.
Boolean/integer fields accept strings to support env var expansion.

Add schema_test.go with four validation tests:
- fixture validation against all testdata/book/*.yml
- reserved key coverage check against code constants
- runner config field coverage via reflect
- invalid runbook rejection

Remove unused useAgent field from sshd_local_forward_with_openapi3.yml
as it has no corresponding implementation in sshRunnerConfig.
@k1LoW k1LoW self-assigned this Feb 18, 2026
@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @k1LoW, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the runn project by introducing a robust JSON Schema for its runbook YAML format. This schema provides clear definitions for runbook structure, improving maintainability and enabling automated validation. Accompanying the schema are comprehensive test suites that ensure its accuracy and coverage, validating against existing configurations and preventing common errors. This change streamlines the development and usage of runbooks by providing a standardized and verifiable format.

Highlights

  • JSON Schema for Runbook YAML: A new JSON Schema (Draft 2020-12) has been added to define the structure and validate runn runbook YAML files, covering all top-level properties, runner configurations, step definitions, and polymorphic types.
  • Flexible Type Handling: Boolean and integer fields in the schema now accept string types, enabling support for environment variable expansion without validation errors.
  • Comprehensive Schema Validation Tests: New Go tests (schema_test.go) have been introduced to ensure the schema's correctness, including validating against existing runbook fixtures, checking coverage of reserved keys, verifying runner configuration fields via reflection, and rejecting invalid runbook structures.
  • Test Fixture Refinement: An unused useAgent field was removed from sshd_local_forward_with_openapi3.yml as it lacked a corresponding implementation in the sshRunnerConfig.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • runbook.schema.json
    • Added a new JSON Schema (Draft 2020-12) for runn runbook YAML files.
  • schema_test.go
    • Added new Go tests to validate the runn runbook JSON Schema.
    • Implemented fixture validation against all testdata/book/*.yml files.
    • Included checks for reserved key coverage against code constants.
    • Added tests for runner configuration field coverage using reflection.
    • Introduced tests to ensure invalid runbooks are correctly rejected by the schema.
  • testdata/book/sshd_local_forward_with_openapi3.yml
    • Removed the useAgent field, which was unused and lacked implementation.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a comprehensive JSON Schema for the runbook YAML format, significantly improving validation and maintainability. The accompanying Go tests (schema_test.go) provide good coverage for the schema's correctness, including fixture validation, reserved key checks, and runner configuration field coverage. The removal of the unused useAgent field in sshd_local_forward_with_openapi3.yml is a positive cleanup. Overall, this is a valuable addition that enhances the robustness and clarity of runbook definitions.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds comprehensive JSON Schema support for the runn runbook YAML format to enable editor validation and autocompletion. The schema covers all top-level properties, runner configurations, step definitions, and polymorphic types. Boolean and integer fields accept strings to support environment variable expansion.

Changes:

  • Added runbook.schema.yaml with complete JSON Schema (Draft 2020-12) covering all runbook properties and runner configurations
  • Added comprehensive schema validation tests in schema_test.go including fixture validation, reserved key coverage, runner config field coverage, and invalid runbook rejection
  • Removed unused useAgent field from sshd_local_forward_with_openapi3.yml as it has no implementation
  • Updated README.md with JSON Schema documentation and usage instructions for YAML Language Server integration

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
runbook.schema.yaml Comprehensive JSON Schema definition for runbook YAML format with all properties, runner configs, and validation rules
schema_test.go Four validation tests ensuring schema accuracy and completeness against code and fixtures
testdata/book/sshd_local_forward_with_openapi3.yml Removed unused useAgent field that has no corresponding implementation
README.md Added JSON Schema section with usage instructions for editor integration

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@github-actions

This comment has been minimized.

@github-actions
Copy link
Contributor

BenchmarkManyRunbooks-4

main (-) #1398 (883028b) +/-
Number of iterations 1 1 0
Nanoseconds per iteration 3,615,815,052 ns/op 3,813,529,630 ns/op 197,714,578 ns/op
Bytes allocated per iteration 1,381,560,472 B/op 1,381,956,112 B/op 395,640 B/op
Allocs per iteration 19,289,061 allocs/op 19,289,073 allocs/op 12 allocs/op
Metadata
main (-) #1398 (883028b)
goos linux linux
goarch amd64 amd64
pkg github.com/k1LoW/runn github.com/k1LoW/runn
cpu AMD EPYC 7763 64-Core Processor AMD EPYC 7763 64-Core Processor

BenchmarkOpenAPI3-4

main (-) #1398 (883028b) +/-
Number of iterations 1 1 0
Nanoseconds per iteration 1,526,573,746 ns/op 1,525,648,617 ns/op -925,129 ns/op
Bytes allocated per iteration 818,906,928 B/op 819,131,392 B/op 224,464 B/op
Allocs per iteration 11,608,966 allocs/op 11,609,261 allocs/op 295 allocs/op
Metadata
main (-) #1398 (883028b)
goos linux linux
goarch amd64 amd64
pkg github.com/k1LoW/runn github.com/k1LoW/runn
cpu AMD EPYC 7763 64-Core Processor AMD EPYC 7763 64-Core Processor

BenchmarkSingleRunbook-4

main (-) #1398 (883028b) +/-
Number of iterations 22 24 2
Nanoseconds per iteration 46,662,833 ns/op 45,995,019 ns/op -667,814 ns/op
Bytes allocated per iteration 23,052,142 B/op 23,029,569 B/op -22,573 B/op
Allocs per iteration 186,824 allocs/op 186,804 allocs/op -20 allocs/op
Metadata
main (-) #1398 (883028b)
goos linux linux
goarch amd64 amd64
pkg github.com/k1LoW/runn github.com/k1LoW/runn
cpu AMD EPYC 7763 64-Core Processor AMD EPYC 7763 64-Core Processor

Reported by octocov

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@github-actions
Copy link
Contributor

Code Metrics Report

main (9a8a8e6) #1398 (883028b) +/-
Coverage 60.3% 60.3% +0.0%
Code to Test Ratio 1:0.8 1:0.8 +0.0
Test Execution Time 7m19s 7m58s +39s
Details
  |                     | main (9a8a8e6) | #1398 (883028b) |  +/-  |
  |---------------------|----------------|-----------------|-------|
+ | Coverage            |          60.3% |           60.3% | +0.0% |
  |   Files             |             81 |              81 |     0 |
  |   Lines             |           9246 |            9246 |     0 |
+ |   Covered           |           5578 |            5579 |    +1 |
+ | Code to Test Ratio  |          1:0.8 |           1:0.8 |  +0.0 |
  |   Code              |          17298 |           17298 |     0 |
+ |   Test              |          14395 |           14453 |   +58 |
- | Test Execution Time |          7m19s |           7m58s |  +39s |

Code coverage of files in pull request scope (65.2% → 65.9%)

Files Coverage +/- Status
internal/expr/expr.go 65.9% +0.7% affected

Reported by octocov

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Comments