Skip to content

feat: Add poe-command workflow, slash command, and test-my-tools poe task#894

Merged
Aaron ("AJ") Steers (aaronsteers) merged 5 commits intomainfrom
devin/1764979475-add-poe-command-workflow
Dec 17, 2025
Merged

feat: Add poe-command workflow, slash command, and test-my-tools poe task#894
Aaron ("AJ") Steers (aaronsteers) merged 5 commits intomainfrom
devin/1764979475-add-poe-command-workflow

Conversation

@aaronsteers
Copy link
Member

@aaronsteers Aaron ("AJ") Steers (aaronsteers) commented Dec 6, 2025

feat: Add poe-command workflow, slash command, and test-my-tools poe task

Summary

This PR adds infrastructure for running poe commands via GitHub slash commands and a new test-my-tools poe task that invokes Claude Code in headless mode with the MCP server's test-my-tools prompt.

Changes:

  • New workflow (.github/workflows/poe-command.yml): Handles /poe slash commands using the poe-command-processor action
  • Updated slash command dispatch: Added poe to the list of recognized slash commands
  • New poe task (test-my-tools): Invokes Claude Code in headless mode with the MCP server, accepting an optional scope parameter to focus testing

Review & Testing Checklist for Human

  • Verify Claude Code CLI flags: The poe task uses claude --print, --allowedTools, and --mcp-config flags. Confirm these match the actual Claude Code CLI interface for headless mode
  • Verify .mcp.json exists or will be created: The poe task references .mcp.json in the repo root
  • Verify MCP server name pattern: The --allowedTools "mcp__airbyte-mcp__*" pattern should match how the MCP server is registered in the config
  • Verify secrets are configured: The workflow requires OCTAVIA_BOT_APP_ID and OCTAVIA_BOT_PRIVATE_KEY secrets

Recommended test plan:

  1. Ensure org-wide secrets for Claude Code and the GitHub App are configured
  2. Run poe test-my-tools locally to verify the task invokes Claude Code correctly
  3. Test with a scope parameter: poe test-my-tools --scope "focus on read-only tools"
  4. Test the slash command by commenting /poe test-my-tools on a PR

Notes

This is part of a coordinated change across three repos (PyAirbyte, connector-builder-mcp, airbyte-ops-mcp) to add poe commands and slash commands for the test-my-tools prompt.

Requested by: AJ Steers (Aaron ("AJ") Steers (@aaronsteers))
Link to Devin run: https://app.devin.ai/sessions/856354e3548a42e4bc602f3db88fac65

Summary by CodeRabbit

  • New Features

    • Added a configurable testing task to run headless code checks with an optional scope parameter.
  • Chores

    • Added a manual, on-demand workflow to trigger task processing with authenticated execution.
    • Expanded supported slash commands for workflow dispatch.
    • Added CI configuration and updated CI requirements to include an additional environment credential and side‑car Docker support.

✏️ Tip: You can customize this high-level summary in your review settings.

Important

Auto-merge enabled.

This PR is set to merge automatically when all requirements are met.

devin-ai-integration bot and others added 2 commits December 6, 2025 00:05
Co-Authored-By: AJ Steers <aj@airbyte.io>
@devin-ai-integration
Copy link
Contributor

Original prompt from AJ Steers
Received message in Slack channel #ask-devin-ai:

@Devin - Investigate the new "test-tools" prompt in Airbyte Ops MCP. Create a pair of PRs to repeat this pattern in Connector Builder MCP and PyAirbyte MCP? In PyAirbyte, we'll need an optional prompt input like 'scope' that will (optionally) focus tests further than just "test everything".
Thread URL: https://airbytehq-team.slack.com/archives/C08BHPUMEPJ/p1764958740107279

@devin-ai-integration
Copy link
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@github-actions
Copy link

github-actions bot commented Dec 6, 2025

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

Testing This PyAirbyte Version

You can test this version of PyAirbyte using the following:

# Run PyAirbyte CLI from this branch:
uvx --from 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1764979475-add-poe-command-workflow' pyairbyte --help

# Install PyAirbyte from this branch for development:
pip install 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1764979475-add-poe-command-workflow'

Helpful Resources

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

  • /fix-pr - Fixes most formatting and linting issues
  • /poetry-lock - Updates poetry.lock file
  • /test-pr - Runs tests with the updated PyAirbyte

Community Support

Questions? Join the #pyairbyte channel in our Slack workspace.

📝 Edit this welcome message.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Dec 6, 2025

📝 Walkthrough

Walkthrough

Adds a manual GitHub Actions workflow to run the Poe Slash Command Processor with GitHub App authentication, extends the slash-command dispatcher to accept poe, adds a Poe task test-my-tools to pyproject.toml, and adds an MCP CI JSON configuration plus small airbyte_ci config additions. (50 words)

Changes

Cohort / File(s) Summary
New GitHub Actions workflow
.github/workflows/poe-command.yml
Adds a manual ("workflow_dispatch") workflow named "On-Demand Poe Task" that authenticates as a GitHub App (using a dedicated token) and runs the Poe Slash Command Processor action with inputs for PR number and optional comment-id.
Slash command dispatcher update
.github/workflows/slash_command_dispatch.yml
Extends the dispatch workflow's allowed commands to include the poe command in the commands list; no other behavioral changes detected.
Poe task + airbyte_ci changes
pyproject.toml
Adds [tool.poe.tasks.test-my-tools] — a public Poe task running Claude Code in headless mode with an MCP prompt and exposes an optional scope argument. Also adds to [tool.airbyte_ci]: required_environment_variables = ["GCP_GSM_CREDENTIALS"] and side_car_docker_engine = true.
MCP CI configuration
.github/mcp-ci-tests.mcp.json
Adds an MCP configuration launching airbyte-mcp via poetry run airbyte-mcp, with environment variables GCP_GSM_CREDENTIALS and AIRBYTE_CLOUD_MCP_SAFE_MODE=1 set for CI.

Sequence Diagram(s)

mermaid
sequenceDiagram
autonumber
participant Maintainer as Maintainer (manual trigger)
participant GH as GitHub Actions
participant App as GitHub App (auth)
participant Runner as Actions Runner
participant Poe as Poe Slash Command Processor
participant PR as Pull Request

Maintainer->>GH: Trigger "On-Demand Poe Task" (workflow_dispatch)
GH->>App: Request installation token (app authentication)
App-->>GH: Return GitHub App token
GH->>Runner: Start job with token + inputs (pr_number, comment_id?)
Runner->>Poe: Invoke Poe Slash Command Processor action (with token & inputs)
Poe->>PR: Read PR/comment context via API (using token)
Poe-->>Runner: Produce outputs/logs
Runner-->>GH: Job result (success/failure)

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

  • Please verify the GitHub App permissions and token scope in .github/workflows/poe-command.yml follow least-privilege practices — wdyt?
  • Can we confirm the Poe action inputs (PR number, optional comment-id) and how the token is passed/secreted are correct for the intended runner context — wdyt?
  • Could we validate the pyproject.toml Claude Code / MCP invocation and the mcp__airbyte-mcp__* tool restriction syntax match CI/tooling expectations — wdyt?

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes all three main additions: a poe-command workflow, slash command integration, and the test-my-tools poe task, directly matching the changeset.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch devin/1764979475-add-poe-command-workflow

📜 Recent review details

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between bd7b5aa and 5aaaa96.

📒 Files selected for processing (2)
  • .github/workflows/slash_command_dispatch.yml (1 hunks)
  • pyproject.toml (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
  • .github/workflows/slash_command_dispatch.yml
  • pyproject.toml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
  • GitHub Check: Pytest (Fast)
  • GitHub Check: Pytest (All, Python 3.10, Ubuntu)
  • GitHub Check: Pytest (All, Python 3.11, Ubuntu)
  • GitHub Check: Pytest (No Creds)

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 69f6c96 and 188e643.

📒 Files selected for processing (3)
  • .github/workflows/poe-command.yml (1 hunks)
  • .github/workflows/slash_command_dispatch.yml (1 hunks)
  • pyproject.toml (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (6)
  • GitHub Check: Pytest (All, Python 3.11, Windows)
  • GitHub Check: Pytest (All, Python 3.10, Windows)
  • GitHub Check: Pytest (All, Python 3.11, Ubuntu)
  • GitHub Check: Pytest (All, Python 3.10, Ubuntu)
  • GitHub Check: Pytest (Fast)
  • GitHub Check: Pytest (No Creds)
🔇 Additional comments (4)
.github/workflows/slash_command_dispatch.yml (1)

33-37: Clean addition to the slash command dispatcher.

Adding "poe" to the allowed commands looks good and follows the existing pattern. It integrates naturally with the GitHub App authentication and will properly route to the on-demand Poe task workflow.

.github/workflows/poe-command.yml (2)

22-22: Pin the GitHub App token action to a commit SHA for consistency and security.

I notice this workflow uses actions/create-github-app-token@v2 without a commit SHA pin, while the slash_command_dispatch.yml pins the same action to @67018539274d69449ef7c02e8e71183d1719ab42 # v2.1.4. Would you consider using the same pinned version here for consistency and to prevent unexpected version drift?


31-35: Verify the third-party Poe command processor action.

The workflow delegates all execution logic to aaronsteers/poe-command-processor@v1.3.3. Before merging, can you confirm:

  • Is this action maintained by the Airbyte team or a trusted community maintainer?
  • Does it have reasonable activity/maintenance?
  • Are you confident in its security and stability?

This helps ensure the workflow remains reliable and trustworthy.

pyproject.toml (1)

165-169: Verify the .mcp.json configuration file exists and is documented.

The task references .mcp.json config file, but I don't see it in the PR files provided. Can you confirm:

  • Does .mcp.json exist in the repository root?
  • Is there documentation explaining what this file contains and how to set it up?
  • Should it be committed to the repo or created locally by users?

This will help ensure the task is usable for other developers.

@github-actions
Copy link

github-actions bot commented Dec 6, 2025

PyTest Results (Fast Tests Only, No Creds)

348 tests  ±0   348 ✅ ±0   5m 49s ⏱️ +3s
  1 suites ±0     0 💤 ±0 
  1 files   ±0     0 ❌ ±0 

Results for commit 5aaaa96. ± Comparison against base commit 9c9f4a7.

♻️ This comment has been updated with latest results.

…e task

Co-Authored-By: AJ Steers <aj@airbyte.io>
@github-actions
Copy link

github-actions bot commented Dec 6, 2025

PyTest Results (Full)

416 tests  ±0   399 ✅ ±0   24m 30s ⏱️ + 1m 18s
  1 suites ±0    17 💤 ±0 
  1 files   ±0     0 ❌ ±0 

Results for commit 5aaaa96. ± Comparison against base commit 9c9f4a7.

♻️ This comment has been updated with latest results.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
pyproject.toml (1)

177-177: Consider scoping GCP_GSM_CREDENTIALS to task-specific configuration rather than requiring it globally.

Adding required_environment_variables = ["GCP_GSM_CREDENTIALS"] marks this credential as mandatory for all CI runs. However, the codebase already handles missing credentials gracefully—integration tests skip with pytest.skip() if unavailable, and MCP utilities check availability before initialization.

Since only specific workflows (integration tests and MCP features) actually need this credential, would it be better to configure it as a task-specific requirement rather than enforcing it globally? This would avoid breaking CI workflows that don't require GCP Secret Manager access, wdyt?

📜 Review details

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between b44abc5 and bd7b5aa.

📒 Files selected for processing (2)
  • .github/mcp-ci-tests.mcp.json (1 hunks)
  • pyproject.toml (1 hunks)
✅ Files skipped from review due to trivial changes (1)
  • .github/mcp-ci-tests.mcp.json

@aaronsteers Aaron ("AJ") Steers (aaronsteers) merged commit ce2970d into main Dec 17, 2025
25 checks passed
@aaronsteers Aaron ("AJ") Steers (aaronsteers) deleted the devin/1764979475-add-poe-command-workflow branch December 17, 2025 07:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant