Skip to content

Chore: Add workflow to detect suspicious comments and PRs from forks#1746

Merged
eleanorreem merged 1 commit intodevelopfrom
Chore/Add-workflow-for-AI-bot-detection
Mar 17, 2026
Merged

Chore: Add workflow to detect suspicious comments and PRs from forks#1746
eleanorreem merged 1 commit intodevelopfrom
Chore/Add-workflow-for-AI-bot-detection

Conversation

@bl00dymarie
Copy link
Contributor

Resolves no open issue.

What changes did you make and why did you make them?

For our open source community involvement we want to identify AI bots that create

  1. issue comments
  2. PRs from forks

I chatted with Copilot to create a Gh workflow that helps us detect some common patterns and attach labels to suspicious PRs and adds a report to a suspicious PR or issue comment.

How it works

The workflow calculates a score for PRs and based on that adds a label: requires-manual-review or 🚩-suspicious-fork.

🤖 It checks the accounts from the PRs from forks and from issue commenters whether it seems to be a bot by checking patterns such as:

  • Very new account (< 7 days)
  • Zero followers and minimal repos
  • Fork created very recently (< 24 hours)
  • Bot-like username patterns
    and a few more checks.

🔢 In PRs it also checks for these commit patterns:

  • Very low commit message diversity
  • Generic/repetitive commit messages
  • Suspiciously consistent timing intervals

💬 In issue comments it also checks the comment for:

  • AI service indicators
  • Suspiciously perfect or generic response
  • Copy-paste indicators
  • Same comment repeated on multiple issues

Did you run tests?

No, because this is just a Github Workflow file.

@vercel
Copy link

vercel bot commented Mar 16, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
bloom-frontend Ready Ready Preview, Comment Mar 16, 2026 1:48pm

Request Review

@bl00dymarie bl00dymarie self-assigned this Mar 16, 2026
@eleanorreem eleanorreem merged commit 8090f90 into develop Mar 17, 2026
10 checks passed
@eleanorreem eleanorreem deleted the Chore/Add-workflow-for-AI-bot-detection branch March 17, 2026 14:06
@cypress
Copy link

cypress bot commented Mar 17, 2026

Bloom frontend    Run #1583

Run Properties:  status check passed Passed #1583  •  git commit 8090f90543: Chore: Add workflow to detect suspicious comments and PRs from forks
Project Bloom frontend
Branch Review develop
Run status status check passed Passed #1583
Run duration 04m 24s
Commit git commit 8090f90543: Chore: Add workflow to detect suspicious comments and PRs from forks
Committer bl00dymarie
View all properties for this run ↗︎

Test results
Tests that failed  Failures 0
Tests that were flaky  Flaky 0
Tests that did not run due to a developer annotating a test with .skip  Pending 12
Tests that did not run due to a failure in a mocha hook  Skipped 0
Tests that passed  Passing 102
⚠️ You've recorded test results over your free plan limit.
Upgrade your plan to view test results.
View all changes introduced in this branch ↗︎

@github-actions
Copy link
Contributor

🔍 Bot Detection Alert: Flagged Comment

Suspicion Score: 35/100
Commenter: @cypress[bot]

Account Analysis (Score: 25)

  • Bot-like username: "cypress[bot]"

Comment Content Analysis (Score: 10)

  • Comment may be auto-generated or templated

⚠️ Note: This comment appears to be from a bot or AI assistant. Please verify its relevance and accuracy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants