-
Notifications
You must be signed in to change notification settings - Fork 75
OSS-CRS TI Sandbox Application #559
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
GeauxJD
wants to merge
2
commits into
ossf:main
Choose a base branch
from
GeauxJD:add-oss-crs-app
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
132 changes: 132 additions & 0 deletions
132
process/project-lifecycle-documents/oss-crs_sandbox_stage.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,132 @@ | ||
| # OSS-CRS TI Sandbox Application | ||
|
|
||
| ## List of project maintainers (alphabetic order) | ||
|
|
||
| * Andrew Chin, Georgia Institute of Technology, @azchin | ||
| * Cen Zhang, Georgia Institute of Technology, @occia | ||
| * Dongkwan Kim, Georgia Institute of Technology, @0xdkay | ||
| * Fabian Fleischer, Georgia Institute of Technology, @fab1ano | ||
| * Hanqing Zhao, Georgia Institute of Technology, @hq1995 | ||
| * Jiho Kim, Georgia Institute of Technology, @jhkimx2 | ||
| * Taesoo Kim, Georgia Institute of Technology & Microsoft, @tsgates | ||
| * Younggi Park, Independent Researcher, @grill66 | ||
| * Youngjoon Kim, Georgia Institute of Technology, @acorn421 | ||
| * Yu-Fu Fu, Georgia Institute of Technology, @fuyu0425 | ||
|
|
||
| Note: Maintainers may be updated in the future (some can become inactive or other AIxCC team members may join). | ||
|
|
||
| ## Sponsor | ||
|
|
||
| AI / ML Security Working Group | ||
|
|
||
| The project commits to providing quarterly updates on progress to the AI / ML Security Working Group. | ||
|
|
||
| ## Mission of the project | ||
|
|
||
| OSS-CRS (Open Source Software Cyber Reasoning System) provides a standardized infrastructure for building, running, and evaluating Cyber Reasoning Systems (CRS) that perform automated vulnerability discovery and remediation in open source software. | ||
|
|
||
| ### Problem Statement | ||
|
|
||
| Automated vulnerability discovery is critical for securing the open source software supply chain. While fuzzing tools like OSS-Fuzz have proven effective at finding bugs, the ecosystem lacks: | ||
|
|
||
| 1. **Standardized CRS interfaces**: Each vulnerability discovery tool has unique deployment requirements, making comparison and integration difficult | ||
| 2. **Reproducible evaluation frameworks**: No common infrastructure exists for fairly evaluating and benchmarking different CRS approaches | ||
| 3. **LLM integration standards**: As AI-assisted security tools emerge, there is a lack of framework for developing, integrating, and assessing LLM-enhanced security analysis tools | ||
|
|
||
| ### Solution | ||
|
|
||
| OSS-CRS addresses these gaps by providing: | ||
|
|
||
| - **Unified CLI interface** (oss-bugfind-crs / oss-patch-crs run) for any CRS implementation | ||
| - **Docker Compose-based orchestration** for reproducible multi-container deployments | ||
| - **Resource management** with configurable CPU, memory, and LLM budget allocation | ||
| - **LiteLLM integration** for standardized LLM access across different CRS implementations | ||
| - **OSS-Fuzz compatibility** to leverage Google's existing fuzzing infrastructure | ||
| - **Ensemble support** for combining multiple CRS approaches | ||
|
|
||
| ### Alignment with OpenSSF Mission | ||
|
|
||
| OSS-CRS directly supports OpenSSF's mission to secure the open source supply chain by: | ||
|
|
||
| 1. **Democratizing advanced security tooling**: Making sophisticated CRS capabilities accessible through simple CLI commands | ||
| 2. **Enabling fair CRS evaluation**: Providing infrastructure for benchmarking vulnerability discovery tools (CRSBench ecosystem) | ||
| 3. **Accelerating security research**: Lowering the barrier to experiment with and deploy new vulnerability discovery techniques | ||
| 4. **Supporting AI-augmented security**: Providing standards for integrating LLMs into security workflows | ||
|
|
||
| ### Novel Contributions | ||
|
|
||
| OSS-CRS represents a novel approach in the security tooling space: | ||
|
|
||
| - First open infrastructure for standardized CRS deployment and evaluation | ||
| - Bridges traditional fuzzing (OSS-Fuzz) with modern LLM-augmented approaches | ||
| - Supports both bug-finding and bug-fixing CRS modes | ||
| - Part of the larger CRSBench ecosystem for advancing automated vulnerability research | ||
|
|
||
| ## IP policy and licensing due diligence | ||
|
|
||
| This is a new project developed at Georgia Institute of Technology and will be contributed under MIT license. | ||
|
|
||
| If applicable, the Linux Foundation will conduct IP due diligence upon approval for the Sandbox stage. | ||
|
|
||
| ## Project References | ||
|
|
||
| | Reference | URL | | ||
| |--------------------|--------------------------------------------------------| | ||
| | Repo | https://github.com/sslab-gatech/oss-crs | | ||
| | Website | (To be created upon Sandbox approval) | | ||
| | Contributing guide | (To be created upon Sandbox approval) | | ||
| | Security.md | (To be created upon Sandbox approval) | | ||
| | Roadmap | See below | | ||
| | Demos | See README.md in repository | | ||
| | Other-1 | Bundled repo: [Our Benchmark]( https://github.com/sslab-gatech/CRSBench) | | ||
steiza marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| | Other-2 | Related: [OSS-Fuzz](https://github.com/google/oss-fuzz)| | ||
|
|
||
| Note: Our bundled benchmark repository is still under AIxCC competition organizer review procedure for open-source and we assume its release doesn’t block the Sandbox approval. | ||
|
|
||
| ## Current Capabilities | ||
|
|
||
| - **Bug-finding CRS support**: Build and run CRS implementations for vulnerability discovery | ||
| - **Bug-fixing CRS support**: Infrastructure for automated patch generation | ||
| - **Multiple CRS integrations**: | ||
| - `atlantis-c-libafl` - LibAFL-based fuzzer for C projects | ||
| - `atlantis-c-bullseye` - Directed fuzzer for C projects | ||
| - `atlantis-multilang-dind` - Team Atlanta multilang bug finding module | ||
| - `atlantis-multilang-given_fuzzer` - Team Atlanta multilang version libfuzzer | ||
| - `atlantis-java-main` - Team Atlanta Java bug finding module | ||
| - `atlantis-java-atljazzer` - Team Atlanta Java bug finding module, directed fuzzer | ||
| - `crs-libfuzzer` - Default libfuzzer | ||
| - `atlantis-claude-code` - Team Atlanta patch agent: claude-code-like agent | ||
| - `swe-agent` - Team Atlanta patch agent SWE-Agent | ||
| - `atlantis-vincent` - Team Atlanta patch agent vincent | ||
| - `atlantis-prism` - Team Atlanta patch agent prism | ||
| - `atlantis-multi-retrieval` - Team Atlanta patch agent multi-retrival | ||
| - `42-patch-agent` - Team 42-b3yond-6ug patching module | ||
| - `buttercup-patcher` - Team Trail of Bits patching module | ||
| - Ensemble configurations combining multiple CRS approaches | ||
| - **Resource management**: YAML-based configuration for CPU cores, memory limits, LLM budgets | ||
| - **LiteLLM integration**: Automated proxy deployment for LLM access control | ||
|
|
||
| ### Roadmap | ||
GeauxJD marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| 1/12/26 Roadmap Presentation to CRS SIG meeting: https://docs.google.com/presentation/d/1R9XAF6VOVimdlXL346CnNrMkOzJFVyFhJF9lrYtHnZI/edit?slide=id.g3b67cb90a0c_0_0#slide=id.g3b67cb90a0c_0_0 | ||
|
|
||
| 1. **Infrastructure Hardening** | ||
| - Add SECURITY.md and vulnerability reporting process | ||
| - Create comprehensive CONTRIBUTING.md | ||
| - Improve documentation and user guides | ||
|
|
||
| 2. **CRS Ecosystem Expansion** | ||
| - Support additional CRSes from AIxCC teams and community | ||
| - Add more language support beyond C and Java | ||
| - Integrate with additional LLM providers | ||
|
|
||
| 3. **Evaluation Framework** | ||
| - Standardized benchmark suite integration | ||
| - Automated metrics collection and reporting | ||
| - CI/CD integration for CRS testing | ||
|
|
||
| 4. **Community Building** | ||
| - Public issue tracker and discussion forums | ||
| - Regular community calls | ||
| - Integration with other OpenSSF security tooling projects | ||
|
|
||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Our criteria is: "Projects must have a minimum of three maintainers with a minimum of two different organization affiliations." This really looks borderline...