Skip to content

Commit 4f56af5

Browse files
JacobMagesHaskinsContrastJacobMagesHaskinsContrast
authored
Merge pull request #76 from Contrast-Security-OSS/AIML-180_contrast_llm_documentation
AIML-180: Add Contrast LLM Early Access documentation
2 parents 7bdc611 + f02247a commit 4f56af5

File tree

3 files changed

+371
-3
lines changed

3 files changed

+371
-3
lines changed

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ Please follow the specific setup instructions link for the coding agent of your
3434
## Key Features
3535

3636
* **Support for Multiple Coding Agents**: Choose to use either the internal SmartFix coding agent or GitHub Copilot to remediate your project's vulnerabilities
37-
* **Contrast LLM or Bring Your Own LLM (BYOLLM):** Use Contrast's LLM service for seamless setup, or for advanced users who prefer their own LLM provider use the optional Bring Your Own LLM (BYOLLM) configuration.
37+
* **Contrast LLM or Bring Your Own LLM (BYOLLM):** For the SmartFix Coding Agent, use Contrast's managed LLM service for seamless setup with enterprise-grade security, or for advanced users who prefer their own LLM provider use the optional Bring Your Own LLM (BYOLLM) configuration. [Learn more about Contrast LLM](docs/contrast_llm_early_access.md). Note: Contrast LLM is only available with the SmartFix Coding Agent.
3838
* **Configurable PR Throttling:** Control the volume of automated PRs using `max_open_prs`.
3939
* **Debug Mode:** Enable `debug_mode: 'true'` for verbose logging in the GitHub Action output.
4040

‎docs/contrast_llm_early_access.md‎

Lines changed: 366 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,366 @@
1+
# Contrast LLM Early Access - SmartFix Integration
2+
3+
## Legal Disclaimer
4+
5+
When you use Contrast AI SmartFix with Contrast LLM, you agree that your code and other data will be submitted to Contrast's managed LLM service. All data processing and AI model usage is governed by your existing Contrast Security service agreement and data processing terms.
6+
7+
## Overview
8+
9+
**Contrast LLM** is a secure, sandboxed, Contrast-hosted LLM that the SmartFix coding agent can use. It uses your existing Contrast API keys so there is no additional LLM configuration required.
10+
11+
**Important**: Contrast LLM is only available with the SmartFix Coding Agent. It does not work with GitHub Copilot or Claude Code agent integrations, which use their respective external services.
12+
13+
### Key Benefits
14+
15+
- **Minimal LLM Configuration**: Uses existing Contrast API keys. No additional LLM provider configuration required
16+
- **Free During Beta**: Contrast absorbs all LLM costs during early access program
17+
- **Enterprise-Grade Security**: All AI processing occurs within Contrast's secure infrastructure using a sandboxed LLM that doesn't further train on your Contrast data
18+
- **Managed Scaling**: Automatic handling of rate limits, retries, and load balancing
19+
- **Cost Transparency**: Built-in credit tracking and usage monitoring
20+
- **Seamless Integration**: Works out-of-the-box with existing SmartFix workflows
21+
22+
## Early Access Program
23+
24+
Contrast LLM is available as a **free early access beta** to help customers evaluate SmartFix without LLM setup complexity.
25+
26+
### Program Details
27+
- **Cost**: **Free to customers** during beta period
28+
- **Duration**: 6-week evaluation period
29+
- **Credits**: 50 credits per organization to start. A credit represents an opened PR, i.e. a complete remediation attempt.
30+
- **Support**: Dedicated support throughout the evaluation period
31+
32+
### How to Join
33+
Contact your Contrast representative or Customer Success Manager to:
34+
- Confirm eligibility for the early access program
35+
- Schedule onboarding and setup assistance
36+
- Plan feedback sessions to optimize your experience
37+
38+
### What We Ask
39+
To ensure a successful evaluation, participants are expected to:
40+
- Actively test SmartFix with Contrast LLM during the 6-week period
41+
- Provide feedback on the experience and credit usage patterns
42+
- Participate in brief check-in meetings to address any issues
43+
44+
## Quick Start
45+
46+
To use Contrast LLM with the SmartFix Coding Agent, simply set one configuration parameter in the SmartFix action's YAML file (likely `.github/workflows/smartfix.yml`):
47+
48+
```yaml
49+
- name: Run Contrast AI SmartFix - Generate Fixes Action
50+
uses: Contrast-Security-OSS/contrast-ai-smartfix-action@v1
51+
with:
52+
# Enable Contrast LLM (replaces all BYOLLM configuration)
53+
use_contrast_llm: true # <--- Set this config value
54+
55+
# Standard Contrast configuration (unchanged)
56+
contrast_host: ${{ vars.CONTRAST_HOST }}
57+
contrast_org_id: ${{ vars.CONTRAST_ORG_ID }}
58+
contrast_app_id: ${{ vars.CONTRAST_APP_ID }}
59+
contrast_authorization_key: ${{ secrets.CONTRAST_AUTHORIZATION_KEY }}
60+
contrast_api_key: ${{ secrets.CONTRAST_API_KEY }}
61+
62+
# GitHub and build configuration (unchanged)
63+
github_token: ${{ secrets.GITHUB_TOKEN }}
64+
build_command: 'mvn clean install'
65+
# ... other standard settings
66+
```
67+
68+
### Migration from BYOLLM ("Bring Your Own LLM")
69+
70+
If you're currently using the SmartFix Coding Agent with BYOLLM configuration, migrating to Contrast LLM requires only these changes to the SmartFix action's YAML file (likely `.github/workflows/smartfix.yml`):
71+
72+
**Remove these BYOLLM parameters:**
73+
- `agent_model`
74+
- `anthropic_api_key`
75+
- `aws_bearer_token_bedrock`
76+
- `aws_region`
77+
- `gemini_api_key`
78+
- Any AWS credential configuration steps
79+
80+
**Add this single parameter:**
81+
- `use_contrast_llm: true`
82+
83+
### Mixed Configuration Support
84+
85+
**Key Point**: The same customer can use both Contrast LLM and BYOLLM simultaneously across different repositories:
86+
87+
- **Repo 1**: Configured with Contrast LLM → consumes credits
88+
- **Repo 2**: Configured with BYOLLM → does NOT consume credits
89+
90+
This flexibility allows customers to gradually migrate or use different configurations based on repository requirements.
91+
92+
## Configuration Reference
93+
94+
### Required Parameters
95+
96+
| Parameter | Description | Example |
97+
|-----------|-------------|---------|
98+
| `use_contrast_llm` | Enable Contrast LLM service | `true` |
99+
100+
### Standard Contrast Parameters (Unchanged)
101+
102+
All existing Contrast API parameters remain the same:
103+
- `contrast_host`
104+
- `contrast_org_id`
105+
- `contrast_app_id`
106+
- `contrast_authorization_key`
107+
- `contrast_api_key`
108+
109+
## Complete Workflow Example
110+
111+
```yaml
112+
name: Contrast AI SmartFix (Contrast LLM)
113+
114+
on:
115+
pull_request:
116+
types: [closed]
117+
schedule:
118+
- cron: '0 0 * * *' # Daily at midnight UTC
119+
workflow_dispatch:
120+
121+
permissions:
122+
contents: write
123+
pull-requests: write
124+
issues: write
125+
126+
jobs:
127+
generate_fixes:
128+
name: Generate Fixes
129+
runs-on: ubuntu-latest
130+
if: github.event_name == 'workflow_dispatch' || github.event_name == 'schedule'
131+
steps:
132+
- name: Checkout repository
133+
uses: actions/checkout@v4
134+
with:
135+
fetch-depth: 0
136+
137+
- name: Run Contrast AI SmartFix - Generate Fixes Action
138+
uses: Contrast-Security-OSS/contrast-ai-smartfix-action@v1
139+
with:
140+
# --- Contrast LLM Configuration ---
141+
use_contrast_llm: true
142+
143+
# --- Standard Configuration ---
144+
max_open_prs: 5
145+
base_branch: main
146+
build_command: 'mvn clean test'
147+
max_qa_attempts: 6
148+
vulnerability_severities: '["CRITICAL","HIGH","MEDIUM"]'
149+
formatting_command: 'mvn spotless:apply'
150+
151+
# --- GitHub Token ---
152+
github_token: ${{ secrets.GITHUB_TOKEN }}
153+
154+
# --- Contrast API Credentials ---
155+
contrast_host: ${{ vars.CONTRAST_HOST }}
156+
contrast_org_id: ${{ vars.CONTRAST_ORG_ID }}
157+
contrast_app_id: ${{ vars.CONTRAST_APP_ID }}
158+
contrast_authorization_key: ${{ secrets.CONTRAST_AUTHORIZATION_KEY }}
159+
contrast_api_key: ${{ secrets.CONTRAST_API_KEY }}
160+
161+
# --- Optional Configuration ---
162+
skip_writing_security_test: false
163+
debug_mode: ${{ vars.DEBUG_MODE || 'false' }}
164+
165+
# PR handling jobs remain unchanged
166+
handle_pr_merge:
167+
name: Handle PR Merge
168+
runs-on: ubuntu-latest
169+
if: github.event.pull_request.merged == true && contains(join(github.event.pull_request.labels.*.name), 'contrast-vuln-id:VULN-')
170+
steps:
171+
- name: Checkout repository
172+
uses: actions/checkout@v4
173+
with:
174+
fetch-depth: 0
175+
176+
- name: Notify Contrast on PR Merge
177+
uses: Contrast-Security-OSS/contrast-ai-smartfix-action@v1
178+
with:
179+
run_task: merge
180+
github_token: ${{ secrets.GITHUB_TOKEN }}
181+
contrast_host: ${{ vars.CONTRAST_HOST }}
182+
contrast_org_id: ${{ vars.CONTRAST_ORG_ID }}
183+
contrast_app_id: ${{ vars.CONTRAST_APP_ID }}
184+
contrast_authorization_key: ${{ secrets.CONTRAST_AUTHORIZATION_KEY }}
185+
contrast_api_key: ${{ secrets.CONTRAST_API_KEY }}
186+
skip_comments: ${{ vars.SKIP_COMMENTS || 'false' }}
187+
debug_mode: ${{ vars.DEBUG_MODE || 'false' }}
188+
env:
189+
GITHUB_EVENT_PATH: ${{ github.event_path }}
190+
191+
handle_pr_closed:
192+
name: Handle PR Close
193+
runs-on: ubuntu-latest
194+
if: github.event.pull_request.merged == false && contains(join(github.event.pull_request.labels.*.name), 'contrast-vuln-id:VULN-')
195+
steps:
196+
- name: Checkout repository
197+
uses: actions/checkout@v4
198+
with:
199+
fetch-depth: 0
200+
201+
- name: Notify Contrast on PR Closed
202+
uses: Contrast-Security-OSS/contrast-ai-smartfix-action@v1
203+
with:
204+
run_task: closed
205+
github_token: ${{ secrets.GITHUB_TOKEN }}
206+
contrast_host: ${{ vars.CONTRAST_HOST }}
207+
contrast_org_id: ${{ vars.CONTRAST_ORG_ID }}
208+
contrast_app_id: ${{ vars.CONTRAST_APP_ID }}
209+
contrast_authorization_key: ${{ secrets.CONTRAST_AUTHORIZATION_KEY }}
210+
contrast_api_key: ${{ secrets.CONTRAST_API_KEY }}
211+
env:
212+
GITHUB_EVENT_PATH: ${{ github.event_path }}
213+
```
214+
215+
## Technical Architecture
216+
217+
### Supported Models
218+
The service currently provides access to:
219+
- **Anthropic Claude Sonnet 4.5**
220+
221+
### Data Security
222+
- All code and vulnerability data remains within Contrast's secure infrastructure
223+
- No data is shared with external LLM providers
224+
- Processing occurs in SOC 2 Type II compliant environments
225+
- Data is encrypted in transit and at rest
226+
227+
## Comparison: Contrast LLM vs BYOLLM
228+
229+
| Feature | Contrast LLM | BYOLLM |
230+
|---------|--------------|---------|
231+
| **Setup Complexity** | Single parameter | Multiple API keys, credentials |
232+
| **Model Management** | Managed by Contrast | Customer managed |
233+
| **Cost Tracking** | Built-in credit system | Customer's cloud billing |
234+
| **Security** | Contrast infrastructure | Customer's cloud security |
235+
| **Rate Limits** | Managed automatically | Customer configured |
236+
| **Model Updates** | Automatic | Manual migration required |
237+
| **Support** | Contrast support | Customer self-service |
238+
239+
## Troubleshooting
240+
241+
### Common Issues
242+
243+
**1. Credit Exhaustion**
244+
```
245+
Credits have been exhausted. Contact your CSM to request additional credits.
246+
```
247+
**Solution**: Contact your Customer Success Manager to request additional credits during the beta program.
248+
249+
**2. Service Unavailable**
250+
```
251+
Error: Contrast LLM service temporarily unavailable
252+
```
253+
**Solution**: The service will automatically retry. For persistent issues, check Contrast status page.
254+
255+
**3. Configuration Conflicts**
256+
```
257+
Warning: Both use_contrast_llm and agent_model specified
258+
```
259+
**Solution**: Remove BYOLLM parameters when using `use_contrast_llm: true`.
260+
261+
### Debug Information
262+
263+
Enable detailed logging with:
264+
```yaml
265+
debug_mode: true
266+
```
267+
268+
This provides:
269+
- Credit consumption details per PR generation
270+
- Credit remaining after each operation
271+
- Warning logs when credits are low (5 or fewer remaining)
272+
- Error logs when credits exhausted
273+
- Cost tracking per PR generation
274+
- LLM proxy communication logs
275+
- Model selection information
276+
- Performance metrics
277+
278+
### Usage Tracking
279+
280+
Credit usage and SmartFix activity is automatically logged for your visibility:
281+
- Credit consumption per PR generation
282+
- Credit remaining after each operation
283+
- Logs when credits are low / exhausted
284+
- All activity visible in SmartFix debug logs
285+
286+
### Support
287+
288+
For Contrast LLM Early Access support:
289+
1. **Technical Issues**: Include debug logs and workflow configuration
290+
2. **Credit/Usage Questions**: Contact your Customer Success Manager
291+
3. **Feature Requests**: Submit via normal Contrast support channels
292+
293+
## Migration Guide
294+
295+
### From Existing BYOLLM Configuration
296+
297+
**Step 1**: Backup your current workflow file
298+
299+
**Step 2**: Remove BYOLLM configuration
300+
```yaml
301+
# REMOVE these lines:
302+
# agent_model: 'bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0'
303+
# anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
304+
# aws_bearer_token_bedrock: ${{ secrets.AWS_BEARER_TOKEN_BEDROCK }}
305+
# aws_region: ${{ vars.AWS_REGION }}
306+
307+
# REMOVE this step entirely:
308+
# - name: Configure AWS Credentials
309+
# uses: aws-actions/configure-aws-credentials@v4
310+
# with: ...
311+
```
312+
313+
**Step 3**: Add Contrast LLM configuration
314+
```yaml
315+
# ADD this line:
316+
use_contrast_llm: true
317+
```
318+
319+
**Step 4**: Test with a single vulnerability to verify functionality
320+
321+
### Rollback Plan
322+
323+
To revert to BYOLLM, simply:
324+
1. Set `use_contrast_llm: false`
325+
2. Restore your original LLM provider configuration
326+
3. Re-add any removed credential configuration steps
327+
328+
## Important Notes
329+
330+
### Closed PR Behavior
331+
When you close a SmartFix-generated PR, the system may attempt to regenerate a fix for that vulnerability on the next scheduled run. This will consume additional credits.
332+
333+
To avoid unintended credit usage:
334+
- Only close PRs you don't want SmartFix to retry
335+
- Contact support if you need to permanently exclude specific vulnerabilities
336+
- Monitor your credit usage if you frequently close PRs
337+
338+
We're actively working on enhanced controls for this behavior based on beta feedback.
339+
340+
## Early Access Limitations
341+
342+
- **Model Selection**: Limited to Contrast-managed models (currently Anthropic Claude Sonnet 4.5)
343+
- **Custom Prompting**: Standard SmartFix prompts only
344+
- **Regional Availability**: Initially available in US regions
345+
- **Credit Limits**: Subject to early access allocation limits (50 credits initially)
346+
- **Customer Limit**: Maximum 10 customers in beta program
347+
- **Duration**: 6-week evaluation period per customer
348+
349+
## Data Usage and Privacy
350+
351+
During the early access beta, Contrast collects usage analytics to improve the service:
352+
- Credit usage patterns to optimize allocation
353+
- PR success rates to improve fix quality
354+
- Vulnerability types and languages to enhance model performance
355+
- General usage patterns to refine the user experience
356+
357+
All data collection follows your existing Contrast Security data processing agreement. No code content is used for training external models.
358+
359+
## Feedback and Support
360+
361+
362+
Contact your Customer Success Manager or Contrast representative for beta program participation and feedback scheduling.
363+
364+
---
365+
366+
For complete SmartFix documentation, see the [SmartFix Coding Agent Guide](smartfix_coding_agent.md).

‎docs/smartfix_coding_agent.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -178,11 +178,13 @@ jobs:
178178

179179
### LLM Configuration for the SmartFix Coding Agent
180180

181-
The SmartFix Coding Agent uses Contrast's LLM service for seamless setup with no additional configuration required. For advanced users who prefer to use their own LLM provider, you can enable "Bring Your Own LLM" (BYOLLM) by setting `use_contrast_llm: 'false'` and providing credentials for your preferred LLM provider.
181+
The SmartFix Coding Agent requires an LLM to function. There are 2 options for supplying it with an LLM. The first option is to use Contrast's LLM service (available to Early Access program participants) for seamless setup with no additional configuration required. For advanced users who prefer to use their own LLM provider, you can enable "Bring Your Own LLM" (BYOLLM) by setting `use_contrast_llm: 'false'` and providing credentials for your preferred LLM provider.
182182

183183
#### Seamless: Contrast LLM Service
184184

185-
No additional configuration is needed when using Contrast's LLM service (default behavior with `use_contrast_llm: 'true'`).
185+
**Contrast LLM** is a secure, sandboxed Contrast-hosted LLM that the SmartFix coding agent can use. It uses your existing Contrast API keys so there is no additional LLM configuration required.
186+
187+
To use this option, you can request to be a part of our Early Access program. For complete Contrast LLM setup instructions, see the [Contrast LLM Early Access Guide](contrast_llm_early_access.md).
186188

187189
#### Advanced: Bring Your Own LLM (BYOLLM)
188190

0 commit comments

Comments
 (0)