Migrate llm-judge detector to TrustyAI #17
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Reviewer's GuideThis PR migrates the LLM Judge detector to the TrustyAI registry by refactoring and extending the existing build-and-push GitHub Actions workflow—introducing secure env handling, adding build/scan/publish steps for the LLM Judge image—and updating the KServe servingruntime manifest to reference the new TrustyAI image. Sequence diagram for CI pipeline build and publish of LLM Judge detector imagesequenceDiagram
participant Dev as Developer
participant GitHub as GitHub Actions
participant Trivy as Trivy Scanner
participant Quay as Quay.io Registry
participant GHSec as GitHub Security Tab
participant PR as Pull Request
Dev->>GitHub: Push code / open PR
GitHub->>GitHub: Build LLM Judge Docker image
GitHub->>Trivy: Run security scan on image
Trivy-->>GitHub: Return scan results (SARIF)
GitHub->>Quay: Push image (ci or latest tag)
GitHub->>GHSec: Upload SARIF scan results
GitHub->>PR: Post comment with image link
Class diagram for updated environment variable handling in build-and-push workflowclassDiagram
class BuildAndPushWorkflow {
+env PR_HEAD_SHA
+env GITHUB_REF_NAME
+env QUAY_RELEASE_REPO
+env GITHUB_REF
+env GITHUB_HEAD_REF
+env LLM_JUDGE_IMAGE_NAME
+env TAG
+env EXPIRY_LABEL
+step Build LLM Judge image
+step Push LLM Judge image
+step Trivy scan LLM Judge image
+step Upload SARIF for LLM Judge
}
File-Level Changes
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey @saichandrapandraju - I've reviewed your changes and found some issues that need to be addressed.
Blocking issues:
- Using variable interpolation
${{...}}withgithubcontext data in arun:step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code.githubcontext data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable withenv:to store the data and use the environment variable in therun:script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR". (link) - Trivy scan is configured to always succeed, potentially masking critical vulnerabilities. (link)
General comments:
- Instead of echoing an expiry label directly into the Dockerfile, use Docker’s
--labelflag in the build command to avoid mutating the source file. - Consider adding caching for Docker layers (e.g., via
actions/cacheor buildkit cache) to speed up build times for consecutive runs. - Using
pull_request_targetexposes the workflow to untrusted PR code; consider switching to apull_requesttrigger to avoid running malicious changes with elevated permissions.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- Instead of echoing an expiry label directly into the Dockerfile, use Docker’s `--label` flag in the build command to avoid mutating the source file.
- Consider adding caching for Docker layers (e.g., via `actions/cache` or buildkit cache) to speed up build times for consecutive runs.
- Using `pull_request_target` exposes the workflow to untrusted PR code; consider switching to a `pull_request` trigger to avoid running malicious changes with elevated permissions.
## Individual Comments
### Comment 1
<location> `.github/workflows/build-and-push-judge.yaml:107` </location>
<code_context>
+ PR image build completed successfully!
+
+ 📦 [PR image](https://quay.io/repository/trustyai/guardrails-detector-llm-judge-ci?tab=tags): `quay.io/trustyai/guardrails-detector-llm-judge-ci:${{ github.event.pull_request.head.sha }}`
+ - name: Trivy scan
+ uses: aquasecurity/[email protected]
+ with:
+ scan-type: 'image'
+ image-ref: "${{ env.IMAGE_NAME }}:${{ env.TAG }}"
+ format: 'sarif'
+ output: 'trivy-results.sarif'
+ severity: 'MEDIUM,HIGH,CRITICAL'
+ exit-code: '0'
+ ignore-unfixed: false
+ vuln-type: 'os,library'
</code_context>
<issue_to_address>
Trivy scan is configured to always succeed, potentially masking critical vulnerabilities.
With 'exit-code: 0', the workflow will not fail for HIGH or CRITICAL vulnerabilities, which may result in publishing insecure images. To enforce blocking on these severities, set 'exit-code: 1' or handle scan results explicitly.
</issue_to_address>
<suggested_fix>
<<<<<<< SEARCH
exit-code: '0'
=======
exit-code: '1'
>>>>>>> REPLACE
</suggested_fix>
## Security Issues
### Issue 1
<location> `.github/workflows/build-and-push-judge.yaml:51` </location>
<issue_to_address>
**security (opengrep-rules.yaml.github-actions.security.run-shell-injection):** Using variable interpolation `${{...}}` with `github` context data in a `run:` step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. `github` context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with `env:` to store the data and use the environment variable in the `run:` script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
*Source: opengrep*
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
|
@sourcery-ai review |
There was a problem hiding this comment.
Hey @saichandrapandraju - I've reviewed your changes - here's some feedback:
- The build-and-push workflow has a lot of repeated steps for each detector; consider using a matrix job to DRY up the Docker build, push, and Trivy scan steps.
- You’re echoing common vars like PR_HEAD_SHA and QUAY_RELEASE_REPO in multiple steps; moving them to the job-level env block would reduce repetition and make updates easier.
- Instead of echoing ‘quay.expires-after’ into each Dockerfile in a separate step, you could pass --label to docker build to set the expiry label in one go.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The build-and-push workflow has a lot of repeated steps for each detector; consider using a matrix job to DRY up the Docker build, push, and Trivy scan steps.
- You’re echoing common vars like PR_HEAD_SHA and QUAY_RELEASE_REPO in multiple steps; moving them to the job-level env block would reduce repetition and make updates easier.
- Instead of echoing ‘quay.expires-after’ into each Dockerfile in a separate step, you could pass --label to docker build to set the expiry label in one go.
## Individual Comments
### Comment 1
<location> `.github/workflows/build-and-push.yaml:142` </location>
<code_context>
- 📦 [PR image](https://quay.io/trustyai/guardrails-detector-built-in-ci?tab=tags): `quay.io/trustyai/guardrails-detector-built-in-ci:${{ github.event.pull_request.head.sha }}`
+ 📦 [PR image](https://quay.io/repository/trustyai/guardrails-detector-huggingface-runtime-ci?tab=tags): `quay.io/trustyai/guardrails-detector-huggingface-runtime-ci:$PR_HEAD_SHA`
+ 📦 [PR image](https://quay.io/trustyai/guardrails-detector-built-in-ci?tab=tags): `quay.io/trustyai/guardrails-detector-built-in-ci:$PR_HEAD_SHA`
+ 📦 [PR image](https://quay.io/trustyai/guardrails-detector-llm-judge-ci?tab=tags): `quay.io/trustyai/guardrails-detector-llm-judge-ci:$PR_HEAD_SHA`
- name: Trivy scan
uses: aquasecurity/[email protected]
</code_context>
<issue_to_address>
Clarify the label for the new PR image to distinguish it from others.
Specify a unique label such as 'LLM Judge PR image' for this entry to help users differentiate between the images.
</issue_to_address>
<suggested_fix>
<<<<<<< SEARCH
📦 [PR image](https://quay.io/trustyai/guardrails-detector-llm-judge-ci?tab=tags): `quay.io/trustyai/guardrails-detector-llm-judge-ci:$PR_HEAD_SHA`
=======
📦 [LLM Judge PR image](https://quay.io/trustyai/guardrails-detector-llm-judge-ci?tab=tags): `quay.io/trustyai/guardrails-detector-llm-judge-ci:$PR_HEAD_SHA`
>>>>>>> REPLACE
</suggested_fix>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
…el to docker file + add unique name
|
@sourcery-ai review |
There was a problem hiding this comment.
Hey @saichandrapandraju - I've reviewed your changes and they look great!
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Migrate LLM judge detector to official TrustyAI registry.
Changes
- Addedbuild-and-push-judge.yamlworkflow (adapted from existing HF build & push)build-and-push.yamlto include LLM Judge detectorbuild-and-push.yamlto use an intermediate environment variable withenv:and then use those inrun:andbody:to prevent potential injectionsservingruntime.yamlto usequay.io/trustyai/guardrails-detector-llm-judge:latestCloses: #15
Summary by Sourcery
Migrate the LLM Judge detector to the official TrustyAI container registry and integrate it into the existing CI pipeline with automated build, security scanning, and container registry publishing
Enhancements: