Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/workflows/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Llama Stack uses GitHub Actions for Continuous Integration (CI). Below is a tabl

| Name | File | Purpose |
| ---- | ---- | ------- |
| PR Bot Commands | [bot-trigger.yml](bot-trigger.yml) | Bot command for PR |
| Update Changelog | [changelog.yml](changelog.yml) | Creates PR for updating the CHANGELOG.md |
| API Conformance Tests | [conformance.yml](conformance.yml) | Run the API Conformance test suite on the changes. |
| Installer CI | [install-script-ci.yml](install-script-ci.yml) | Test the installation script |
Expand All @@ -12,10 +13,11 @@ Llama Stack uses GitHub Actions for Continuous Integration (CI). Below is a tabl
| Integration Tests (Replay) | [integration-tests.yml](integration-tests.yml) | Run the integration test suites from tests/integration in replay mode |
| Vector IO Integration Tests | [integration-vector-io-tests.yml](integration-vector-io-tests.yml) | Run the integration test suite with various VectorIO providers |
| Pre-commit | [pre-commit.yml](pre-commit.yml) | Run pre-commit checks |
| Pre-commit Bot | [precommit-trigger.yml](precommit-trigger.yml) | Pre-commit bot for PR |
| Run Pre-commit | [precommit-trigger.yml](precommit-trigger.yml) | Run Pre-commit via PR comment |
| Test Llama Stack Build | [providers-build.yml](providers-build.yml) | Test llama stack build |
| Python Package Build Test | [python-build-test.yml](python-build-test.yml) | Test building the llama-stack PyPI project |
| Integration Tests (Record) | [record-integration-tests.yml](record-integration-tests.yml) | Run the integration test suite from tests/integration |
| Run Snapshot Regeneration | [regenerate-snapshot-trigger.yml](regenerate-snapshot-trigger.yml) | Run Snapshot Regeneration via PR comment |
| Check semantic PR titles | [semantic-pr.yml](semantic-pr.yml) | Ensure that PR titles follow the conventional commit spec |
| Close stale issues and PRs | [stale_bot.yml](stale_bot.yml) | Run the Stale Bot action |
| Test External Providers Installed via Module | [test-external-provider-module.yml](test-external-provider-module.yml) | Test External Provider installation via Python module |
Expand Down
130 changes: 130 additions & 0 deletions .github/workflows/bot-trigger.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
name: PR Bot Commands
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a re-written version of precommit-trigger. I used git mv but because the diff is so large it's registering it as a new file annoyingly. Sorry


run-name: Bot command for PR #${{ github.event.issue.number }}

on:
issue_comment:
types: [created]

jobs:
# Shared setup job for both pre-commit and snapshot regeneration
setup:
if: github.event.issue.pull_request && (contains(github.event.comment.body, '@github-actions run precommit') || contains(github.event.comment.body, '@github-actions regenerate snapshots'))
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
outputs:
authorized: ${{ steps.check_author.outputs.authorized }}
pr_number: ${{ steps.check_author.outputs.pr_number }}
pr_head_ref: ${{ steps.check_author.outputs.pr_head_ref }}
pr_head_sha: ${{ steps.check_author.outputs.pr_head_sha }}
pr_head_repo: ${{ steps.check_author.outputs.pr_head_repo }}
pr_base_ref: ${{ steps.check_author.outputs.pr_base_ref }}
is_fork: ${{ steps.check_author.outputs.is_fork }}
command: ${{ steps.detect_command.outputs.command }}

steps:
- name: Detect command
id: detect_command
run: |
COMMENT="${{ github.event.comment.body }}"
if [[ "$COMMENT" == *"@github-actions run precommit"* ]]; then
echo "command=precommit" >> $GITHUB_OUTPUT
elif [[ "$COMMENT" == *"@github-actions regenerate snapshots"* ]]; then
echo "command=regenerate-snapshots" >> $GITHUB_OUTPUT
fi
- name: Check comment author and get PR details
id: check_author
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
// Get PR details
const pr = await github.rest.pulls.get({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: context.issue.number
});

// Check if commenter has write access or is the PR author
const commenter = context.payload.comment.user.login;
const prAuthor = pr.data.user.login;

let hasPermission = false;

// Check if commenter is PR author
if (commenter === prAuthor) {
hasPermission = true;
console.log(`Comment author ${commenter} is the PR author`);
} else {
// Check if commenter has write/admin access
try {
const permission = await github.rest.repos.getCollaboratorPermissionLevel({
owner: context.repo.owner,
repo: context.repo.repo,
username: commenter
});

const level = permission.data.permission;
hasPermission = ['write', 'admin', 'maintain'].includes(level);
console.log(`Comment author ${commenter} has permission: ${level}`);
} catch (error) {
console.log(`Could not check permissions for ${commenter}: ${error.message}`);
}
}

if (!hasPermission) {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body: `❌ @${commenter} You don't have permission to trigger bot commands. Only PR authors or repository collaborators can run this command.`
});
core.setFailed(`User ${commenter} does not have permission`);
return;
}

// Save PR info for later steps
core.setOutput('pr_number', context.issue.number);
core.setOutput('pr_head_ref', pr.data.head.ref);
core.setOutput('pr_head_sha', pr.data.head.sha);
core.setOutput('pr_head_repo', pr.data.head.repo.full_name);
core.setOutput('pr_base_ref', pr.data.base.ref);
core.setOutput('is_fork', pr.data.head.repo.full_name !== context.payload.repository.full_name);
core.setOutput('authorized', 'true');

- name: React to comment
if: steps.check_author.outputs.authorized == 'true'
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
await github.rest.reactions.createForIssueComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: context.payload.comment.id,
content: 'rocket'
});

pre-commit:
needs: setup
if: needs.setup.outputs.authorized == 'true' && needs.setup.outputs.command == 'precommit'
uses: ./.github/workflows/precommit-trigger.yml
with:
pr_number: ${{ needs.setup.outputs.pr_number }}
pr_head_ref: ${{ needs.setup.outputs.pr_head_ref }}
pr_head_sha: ${{ needs.setup.outputs.pr_head_sha }}
pr_head_repo: ${{ needs.setup.outputs.pr_head_repo }}
is_fork: ${{ needs.setup.outputs.is_fork }}

regenerate-snapshots:
needs: setup
if: needs.setup.outputs.authorized == 'true' && needs.setup.outputs.command == 'regenerate-snapshots'
uses: ./.github/workflows/regenerate-snapshots-trigger.yml
with:
pr_number: ${{ needs.setup.outputs.pr_number }}
pr_head_ref: ${{ needs.setup.outputs.pr_head_ref }}
pr_head_sha: ${{ needs.setup.outputs.pr_head_sha }}
pr_head_repo: ${{ needs.setup.outputs.pr_head_repo }}
is_fork: ${{ needs.setup.outputs.is_fork }}
10 changes: 10 additions & 0 deletions .github/workflows/conformance.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,11 @@ jobs:
with:
fetch-depth: 0

- name: Install dependencies
uses: ./.github/actions/setup-runner
with:
python-version: "3.12"

# Check if we should skip conformance testing due to breaking changes
- name: Check if conformance test should be skipped
id: skip-check
Expand Down Expand Up @@ -137,6 +142,11 @@ jobs:
run: |
oasdiff breaking --fail-on ERR $BASE_SPEC $CURRENT_SPEC --match-path '^/v1/'

# never skip this, instead if a breaking change is properly identified -- we should regenerate our snapshot.
- name: Run Pydantic Model Test
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why don't we run this in pre-commit?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think keeping it in the conformance yml makes sense though so it can get the context of the ! or BREAKING CHANGE: from this workflow. In pre-commit people will just override the snapshots locally and repush.

run: |
uv run --no-sync ./scripts/snapshot-test.sh tests/api/test_pydantic_models.py

# Report when test is skipped
- name: Report skip reason
if: steps.skip-check.outputs.skip == 'true'
Expand Down
Loading
Loading