-
Notifications
You must be signed in to change notification settings - Fork 29
feature: Workflow to release new versions to S3 automatically #970
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
| name: Build and Upload Linux x86_64 | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Configure AWS credentials | ||
| uses: aws-actions/configure-aws-credentials@v4 | ||
| with: | ||
| aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} | ||
| aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} | ||
| aws-region: us-west-2 | ||
|
|
||
| - name: Set environment variables | ||
| run: | | ||
| echo "GIT_BRANCH=master" >> $GITHUB_ENV | ||
| echo "GIT_COMMIT=${GITHUB_SHA}" >> $GITHUB_ENV | ||
| echo "BUILD_TYPE=Release" >> $GITHUB_ENV | ||
| echo "UPLOAD_TO_S3=true" >> $GITHUB_ENV | ||
| - name: Build and Upload | ||
| run: | | ||
| export GIT_URL=https://github.com/${GITHUB_REPOSITORY}.git | ||
| export WORKSPACE=${{ github.workspace }} | ||
| ci/build_linux.sh | ||
| - name: Upload artifacts (for debugging) | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: libsnowflakeclient-linux-x86_64 | ||
| path: artifacts/*.tar.gz | ||
| retention-days: 30 | ||
| if-no-files-found: warn | ||
|
|
||
| build-and-upload-linux-aarch64: |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 29 days ago
To fix this, explicitly declare minimal GITHUB_TOKEN permissions for the workflow. The simplest, least-disruptive way is to add a permissions block at the top (workflow) level, right after the on: block, so it applies to all jobs. None of the shown steps need to write to the repo, manage PRs, or interact with other GitHub APIs, so contents: read is sufficient.
Concretely:
- Edit
.github/workflows/release.yml. - After the
on:section (after line 12), insert:
permissions:
contents: readThis will restrict the automatically provided GITHUB_TOKEN for all jobs in this workflow to read-only access to repository contents, satisfying CodeQL’s requirement while preserving existing functionality.
-
Copy modified lines R14-R16
| @@ -11,6 +11,9 @@ | ||
| required: true | ||
| type: string | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| build-and-upload-linux-x86_64: | ||
| name: Build and Upload Linux x86_64 |
| name: Build and Upload Linux aarch64 | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up QEMU | ||
| uses: docker/setup-qemu-action@v3 | ||
| with: | ||
| platforms: arm64 | ||
|
|
||
| - name: Configure AWS credentials | ||
| uses: aws-actions/configure-aws-credentials@v4 | ||
| with: | ||
| aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} | ||
| aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} | ||
| aws-region: us-west-2 | ||
|
|
||
| - name: Set environment variables | ||
| run: | | ||
| echo "GIT_BRANCH=master" >> $GITHUB_ENV | ||
| echo "GIT_COMMIT=${GITHUB_SHA}" >> $GITHUB_ENV | ||
| echo "BUILD_TYPE=Release" >> $GITHUB_ENV | ||
| echo "UPLOAD_TO_S3=true" >> $GITHUB_ENV | ||
| echo "TARGET_PLATFORM=arm64" >> $GITHUB_ENV | ||
| - name: Build and Upload | ||
| run: | | ||
| export GIT_URL=https://github.com/${GITHUB_REPOSITORY}.git | ||
| export WORKSPACE=${{ github.workspace }} | ||
| ci/build_linux.sh | ||
| - name: Upload artifacts (for debugging) | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: libsnowflakeclient-linux-aarch64 | ||
| path: artifacts/*.tar.gz | ||
| retention-days: 30 | ||
| if-no-files-found: warn | ||
|
|
||
| build-and-upload-macos: |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 29 days ago
In general, fix this by explicitly restricting the GITHUB_TOKEN permissions in the workflow using a permissions block, either at the workflow root (applies to all jobs unless overridden) or per job. Since all shown jobs just check out code, build artifacts, configure AWS, and upload artifacts, they only need read access to repo contents; no write or additional scopes are required.
The best minimal change is to add a workflow-level permissions block right after the name: and before on: in .github/workflows/release.yml:
- Set
permissions: contents: readat the top level so all jobs, includingbuild-and-upload-linux-aarch64(line 50), inherit this restriction. - This does not alter any of the existing steps or their behavior, because they only require reading the repository.
- No imports or additional methods are needed; this is a pure YAML configuration change.
Concretely:
- Edit
.github/workflows/release.ymlaround lines 1–4. - Insert:
between
permissions: contents: read
name: Build and Upload Releaseand theon:block.
-
Copy modified lines R3-R5
| @@ -1,5 +1,8 @@ | ||
| name: Build and Upload Release | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| on: | ||
| push: | ||
| tags: |
| name: Build and Upload macOS | ||
| runs-on: macos-14 | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Install Bash | ||
| run: brew install bash | ||
|
|
||
| - name: Install CMake | ||
| run: brew install cmake | ||
|
|
||
| - name: Configure AWS credentials | ||
| uses: aws-actions/configure-aws-credentials@v4 | ||
| with: | ||
| aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} | ||
| aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} | ||
| aws-region: us-west-2 | ||
|
|
||
| - name: Set environment variables | ||
| run: | | ||
| echo "GIT_BRANCH=master" >> $GITHUB_ENV | ||
| echo "GIT_COMMIT=${GITHUB_SHA}" >> $GITHUB_ENV | ||
| echo "BUILD_TYPE=Release" >> $GITHUB_ENV | ||
| echo "UPLOAD_TO_S3=true" >> $GITHUB_ENV | ||
| - name: Build and Upload | ||
| run: | | ||
| export GIT_URL=https://github.com/${GITHUB_REPOSITORY}.git | ||
| export WORKSPACE=${{ github.workspace }} | ||
| ./ci/build_mac.sh | ||
| - name: Upload artifacts (for debugging) | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: libsnowflakeclient-macos | ||
| path: artifacts/*.tar.gz | ||
| retention-days: 30 | ||
| if-no-files-found: warn | ||
|
|
||
| build-and-upload-windows: |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 29 days ago
In general, the fix is to add a permissions: block that constrains the GITHUB_TOKEN to the minimum needed, either at the root of the workflow (to apply to all jobs) or at each job that needs special permissions. Since none of the shown jobs needs to write to the repository, setting contents: read at the workflow level is sufficient and safest.
The single best fix without changing functionality is: at the top of .github/workflows/release.yml, add a root‑level permissions: block right after the name: line (and before on:). This will apply to all jobs (build-and-upload-linux-x86_64, build-and-upload-linux-aarch64, build-and-upload-macos, build-and-upload-windows, verify-upload, etc.) that do not override permissions. The minimal secure starting point suggested by CodeQL is contents: read, which is enough for actions/checkout@v4 to work and does not interfere with AWS credentials or artifact uploads, since those use secrets and separate services rather than GITHUB_TOKEN.
No additional imports, methods, or other definitions are needed; only the YAML permissions: stanza must be added.
-
Copy modified lines R3-R5
| @@ -1,5 +1,8 @@ | ||
| name: Build and Upload Release | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| on: | ||
| push: | ||
| tags: |
| name: Build and Upload Windows | ||
| runs-on: windows-2022 | ||
| strategy: | ||
| matrix: | ||
| platform: ['x64', 'x86'] | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Install CMake | ||
| shell: cmd | ||
| run: | | ||
| curl -L "https://github.com/Kitware/CMake/releases/download/v3.31.6/cmake-3.31.6-windows-x86_64.zip" -o cmake.zip | ||
| tar -xf cmake.zip | ||
| echo %cd%\cmake-3.31.6-windows-x86_64\bin>> %GITHUB_PATH% | ||
| - name: Configure AWS credentials | ||
| uses: aws-actions/configure-aws-credentials@v4 | ||
| with: | ||
| aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} | ||
| aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} | ||
| aws-region: us-west-2 | ||
|
|
||
| - name: Build and Upload | ||
| shell: cmd | ||
| env: | ||
| PLATFORM: ${{ matrix.platform }} | ||
| BUILD_TYPE: Release | ||
| VS_VERSION: VS17 | ||
| GIT_BRANCH: master | ||
| GIT_COMMIT: ${{ github.sha }} | ||
| UPLOAD_TO_S3: true | ||
| run: ci\build_win.bat | ||
|
|
||
| - name: Upload artifacts (for debugging) | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: libsnowflakeclient-windows-${{ matrix.platform }} | ||
| path: artifacts/*.tar.gz | ||
| retention-days: 30 | ||
| if-no-files-found: warn | ||
|
|
||
| verify-upload: |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 29 days ago
In general, to fix this class of issue you add an explicit permissions: block at the workflow level (affecting all jobs) or at specific jobs, reducing the GITHUB_TOKEN to the minimal scopes required. For pure build/test/upload workflows that don’t need to write to the repo or to PRs, contents: read is usually sufficient.
For this specific file, none of the jobs perform repository write operations via GITHUB_TOKEN; they only check out code, configure AWS using secrets, run build scripts, and upload artifacts to S3 and as build artifacts. actions/checkout@v4 works with contents: read. Therefore the safest and simplest fix is to add a single workflow-level permissions: block with contents: read. This will apply to all jobs (build-and-upload-linux-*, build-and-upload-macos, build-and-upload-windows, and verify-upload) without changing their intended behavior.
Concretely:
- Edit
.github/workflows/release.yml. - Insert a workflow-level
permissions:section after thename:(line 1) and before theon:block (line 3). - Set
contents: readas the only permission. No additional imports or dependencies are required.
-
Copy modified lines R3-R5
| @@ -1,5 +1,8 @@ | ||
| name: Build and Upload Release | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| on: | ||
| push: | ||
| tags: |
| name: Verify S3 Upload | ||
| needs: [build-and-upload-linux-x86_64, build-and-upload-linux-aarch64, build-and-upload-macos, build-and-upload-windows] | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Configure AWS credentials | ||
| uses: aws-actions/configure-aws-credentials@v4 | ||
| with: | ||
| aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} | ||
| aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} | ||
| aws-region: us-west-2 | ||
|
|
||
| - name: Extract version | ||
| id: version | ||
| run: | | ||
| if [[ "${{ github.ref }}" == refs/tags/* ]]; then | ||
| VERSION=${GITHUB_REF#refs/tags/v} | ||
| else | ||
| VERSION="${{ github.event.inputs.version }}" | ||
| fi | ||
| echo "version=$VERSION" >> $GITHUB_OUTPUT | ||
| echo "Checking for version: $VERSION" | ||
| - name: Verify uploads | ||
| run: | | ||
| VERSION="${{ steps.version.outputs.version }}" | ||
| echo "=========================================" | ||
| echo "Verifying LibSFC $VERSION on S3" | ||
| echo "=========================================" | ||
| SUCCESS=true | ||
| echo "" | ||
| echo "Checking darwin (macOS): libsnowflakeclient_darwin_Release-${VERSION}.tar.gz" | ||
| if aws s3 ls s3://sfc-eng-data/dependency/libsnowflakeclient/ | grep "libsnowflakeclient_darwin_Release-${VERSION}.tar.gz"; then | ||
| echo "Darwin found" | ||
| else | ||
| echo "Darwin not found" | ||
| SUCCESS=false | ||
| fi | ||
| echo "" | ||
| echo "Checking linux x86_64: libsnowflakeclient_linux_Release-${VERSION}.tar.gz" | ||
| if aws s3 ls s3://sfc-eng-data/dependency/libsnowflakeclient/ | grep "libsnowflakeclient_linux_Release-${VERSION}.tar.gz"; then | ||
| echo "Linux x86_64 found" | ||
| else | ||
| echo "Linux x86_64 not found" | ||
| SUCCESS=false | ||
| fi | ||
| echo "" | ||
| echo "Checking linux aarch64: libsnowflakeclient_linux_Release-${VERSION}.tar.gz" | ||
| if aws s3 ls s3://sfc-eng-data/dependency-aarch64/libsnowflakeclient/ | grep "libsnowflakeclient_linux_Release-${VERSION}.tar.gz"; then | ||
| echo "Linux aarch64 found" | ||
| else | ||
| echo "Linux aarch64 not found" | ||
| SUCCESS=false | ||
| fi | ||
| echo "" | ||
| echo "Checking windows x64: libsnowflakeclient_win64_vs17_Release-${VERSION}.zip" | ||
| if aws s3 ls s3://sfc-eng-data/dependency/libsnowflakeclient/ | grep "libsnowflakeclient_win64_vs17_Release-${VERSION}.zip"; then | ||
| echo "Windows x64 found" | ||
| else | ||
| echo "Windows x64 not found" | ||
| SUCCESS=false | ||
| fi | ||
| echo "" | ||
| echo "Checking windows x86: libsnowflakeclient_win32_vs17_Release-${VERSION}.zip" | ||
| if aws s3 ls s3://sfc-eng-data/dependency/libsnowflakeclient/ | grep "libsnowflakeclient_win32_vs17_Release-${VERSION}.zip"; then | ||
| echo "Windows x86 found" | ||
| else | ||
| echo "Windows x86 not found" | ||
| SUCCESS=false | ||
| fi | ||
| echo "" | ||
| echo "=========================================" | ||
| if [ "$SUCCESS" = true ]; then | ||
| echo "All platforms verified successfully!" | ||
| echo "=========================================" | ||
| else | ||
| echo "Some platforms missing!" | ||
| echo "=========================================" | ||
| exit 1 | ||
| fi |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 29 days ago
To fix the issue, explicitly define minimal GITHUB_TOKEN permissions for this workflow so that jobs do not inherit potentially broad repository defaults. The safest and simplest approach is to add a top-level permissions: block that applies to all jobs. These jobs only need to read the repository (for actions/checkout) and do not push, modify issues, or interact with pull requests, so contents: read is sufficient. Adding this block immediately under the workflow name: (before on:) keeps behavior unchanged while constraining the token.
Concretely:
- Edit
.github/workflows/release.yml. - After line 1 (
name: Build and Upload Release), insert apermissions:block:contents: read
- No other changes are required; no job-specific blocks are necessary because the root-level block will apply to all jobs that do not override it.
- No imports or additional definitions are needed, since this is purely a YAML configuration change.
-
Copy modified lines R3-R5
| @@ -1,5 +1,8 @@ | ||
| name: Build and Upload Release | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| on: | ||
| push: | ||
| tags: |
| run: | | ||
| if [[ "${{ github.ref }}" == refs/tags/* ]]; then | ||
| VERSION=${GITHUB_REF#refs/tags/v} | ||
| else | ||
| VERSION="${{ github.event.inputs.version }}" | ||
| fi | ||
| echo "version=$VERSION" >> $GITHUB_OUTPUT | ||
| echo "Checking for version: $VERSION" | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Semgrep identified an issue in your code:
Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
To resolve this comment:
✨ Commit Assistant fix suggestion
| run: | | |
| if [[ "${{ github.ref }}" == refs/tags/* ]]; then | |
| VERSION=${GITHUB_REF#refs/tags/v} | |
| else | |
| VERSION="${{ github.event.inputs.version }}" | |
| fi | |
| echo "version=$VERSION" >> $GITHUB_OUTPUT | |
| echo "Checking for version: $VERSION" | |
| env: | |
| VERSION_INPUT: ${{ github.event.inputs.version }} | |
| run: | | |
| if [[ "${{ github.ref }}" == refs/tags/* ]]; then | |
| VERSION=${GITHUB_REF#refs/tags/v} | |
| else | |
| VERSION="$VERSION_INPUT" | |
| fi | |
| echo "version=$VERSION" >> $GITHUB_OUTPUT | |
| echo "Checking for version: $VERSION" |
View step-by-step instructions
- Move the usage of
${{ github.event.inputs.version }}into an environment variable by adding it underenv:in the step. - Reference that environment variable inside your shell script with double quotes, e.g.,
"$VERSION_INPUT", instead of using the GitHub context directly inrun:. - Update the script so that the assignment in the
elseclause usesVERSION="$VERSION_INPUT".
For example:
- Add
VERSION_INPUT: ${{ github.event.inputs.version }}underenv:. - Update the step like this:
run: | if [[ "${{ github.ref }}" == refs/tags/* ]]; then VERSION=${GITHUB_REF#refs/tags/v} else VERSION="$VERSION_INPUT" fi echo "version=$VERSION" >> $GITHUB_OUTPUT echo "Checking for version: $VERSION"
This change prevents user-controlled input from being interpreted directly by the shell, reducing the risk of shell injection vulnerabilities.
💬 Ignore this finding
Reply with Semgrep commands to ignore this finding.
/fp <comment>for false positive/ar <comment>for acceptable risk/other <comment>for all other reasons
Alternatively, triage in Semgrep AppSec Platform to ignore the finding created by run-shell-injection.
You can view more details about this finding in the Semgrep AppSec Platform.
No description provided.