Detect AI-Generated Pull Requests Using AI Agents and LLMs

[Cgarg9]

Description:
To enhance repository security and maintain code authenticity, implement an AI-driven system that checks whether a Pull Request (PR) was generated using AI. The system should leverage AI agents and an LLM of your choice to analyze PR content, comments, and commit messages.

Tasks:

Select an LLM and set up an AI agent to evaluate PRs.
Define criteria for detecting AI-generated code (e.g., syntax patterns, comment structures, repetition).
Integrate the AI agent with GitHub Actions or another CI/CD pipeline for automated checks.
Provide a report or confidence score indicating AI-generated content probability.
Name the notebook ai_generated_pr_checker.ipynb.
Update the README with implementation details and references.

[Steve-Wayne]

Sir do we have to build an ai agent ?? Can lang chain be a option do we have set a criteria on basis on amount of plagiarism

[Cgarg9]

@Steve-Wayne langchain can also be used to build ai agent. Ideally i want to deploy it to the CI/CD so that it performs a plag check as soon as a PR is raised but for now you can build an agent which when provided with a file analyses it and returns a plag report. Set the threshold what you want for testing.

[Cgarg9]

However do keep in mind if its not deployed to CI/CD i cant count it as hard so will merge PR with medium label at best.

[Steve-Wayne]

If I build ai agent with langchain then is it possible to integrate it with ci/cd

[Cgarg9]

@Steve-Wayne i will need to check on that. Ideally there should not be any issues but please check in their docs about deployment.