Why faithfullness is NaN most of the times even when context and answer both makes sense. #897
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: Claude Code | |
| on: | |
| issue_comment: | |
| types: [created] | |
| pull_request_review_comment: | |
| types: [created] | |
| issues: | |
| types: [opened, assigned] | |
| pull_request_review: | |
| types: [submitted] | |
| jobs: | |
| claude: | |
| if: | | |
| (github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude')) || | |
| (github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude')) || | |
| (github.event_name == 'pull_request_review' && contains(github.event.review.body, '@claude')) || | |
| (github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude'))) | |
| runs-on: ubuntu-latest | |
| permissions: | |
| contents: write | |
| pull-requests: write | |
| issues: write | |
| id-token: write | |
| actions: read # Required for Claude to read CI results on PRs | |
| steps: | |
| - name: Checkout repository | |
| uses: actions/checkout@v4 | |
| with: | |
| fetch-depth: 1 | |
| - name: Run Claude Code | |
| id: claude | |
| uses: anthropics/claude-code-action@beta | |
| with: | |
| anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }} | |
| # This is an optional setting that allows Claude to read CI results on PRs | |
| additional_permissions: | | |
| actions: read | |
| # Optional: Specify model (defaults to Claude Sonnet 4, uncomment for Claude Opus 4.1) | |
| # model: "claude-opus-4-1-20250805" | |
| # Optional: Customize the trigger phrase (default: @claude) | |
| # trigger_phrase: "/claude" | |
| # Optional: Trigger when specific user is assigned to an issue | |
| # assignee_trigger: "claude-bot" | |
| # Optional: Allow Claude to run specific commands | |
| # allowed_tools: "Bash(npm install),Bash(npm run build),Bash(npm run test:*),Bash(npm run lint:*)" | |
| # Optional: Add custom instructions for Claude to customize its behavior for your project | |
| # custom_instructions: | | |
| # Follow our coding standards | |
| # Ensure all new code has tests | |
| # Use TypeScript for new files | |
| # Optional: Custom environment variables for Claude | |
| # claude_env: | | |
| # NODE_ENV: test | |