Skip to content

Add matrix values to check current and latest version of AWS #915

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from
+57 −19
Draft

Add matrix values to check current and latest version of AWS #915

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
dff02f6
Add matrix values to check current and latest version of AWS
sfc-gh-jmartinezramirez Apr 11, 2024
55a1fa5
Update reports names to include package version
sfc-gh-jmartinezramirez Apr 11, 2024
bfd8240
Typo environment variable
sfc-gh-jmartinezramirez Apr 12, 2024
3a93cb5
Added latest check for others cloud providers
sfc-gh-jmartinezramirez Apr 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 55 additions & 17 deletions .github/workflows/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ jobs:
matrix:
dotnet: ['net6.0', 'net472', 'net471']
cloud_env: ['AZURE', 'GCP', 'AWS']
pkg_version: ['current', 'latest']
steps:
- name: Checkout code
uses: actions/checkout@v3
Expand All @@ -57,35 +58,46 @@ jobs:
run: |
gpg --quiet --batch --yes --decrypt --passphrase="$PARAMETER_SECRET" \
./.github/workflows/parameters/parameters_${{ matrix.cloud_env }}.json.gpg > Snowflake.Data.Tests/parameters.json
- name: Build Driver
- name: Build Driver Current
if: ${{ matrix.pkg_version == 'current' }}
run: |
cd Snowflake.Data.Tests
dotnet restore
dotnet build -f ${{ matrix.dotnet }}
- name: Build Driver Latest
if: ${{ matrix.pkg_version == 'latest' }}
run: |
cd Snowflake.Data.Tests
dotnet add ./../Snowflake.Data/Snowflake.Data.csproj package AWSSDK.S3
dotnet add ./../Snowflake.Data/Snowflake.Data.csproj package Google.Cloud.Storage.V1
dotnet add ./../Snowflake.Data/Snowflake.Data.csproj package Azure.Storage.Blobs
dotnet restore
dotnet build -f ${{ matrix.dotnet }}
- name: Run Tests
run: |
cd Snowflake.Data.Tests
dotnet-coverage collect "dotnet test --framework ${{ matrix.dotnet }} --no-build -l console;verbosity=normal" --output windows_${{ matrix.dotnet }}_${{ matrix.cloud_env }}_coverage.xml --output-format cobertura --settings coverage.config
dotnet-coverage collect "dotnet test --framework ${{ matrix.dotnet }} --no-build -l console;verbosity=normal" --output windows_${{ matrix.dotnet }}_${{ matrix.cloud_env }}_${{ matrix.pkg_version }}_coverage.xml --output-format cobertura --settings coverage.config
env:
snowflake_cloud_env: ${{ matrix.cloud_env }}
pkg_version: ${{ matrix.pkg_version }}
net_version: ${{ matrix.dotnet }}
- name: Upload Code Coverage Report
uses: actions/upload-artifact@v3
with:
name: code-coverage-report
path: Snowflake.Data.Tests\windows_${{ matrix.dotnet }}_${{ matrix.cloud_env }}_coverage.xml
path: Snowflake.Data.Tests\windows_${{ matrix.dotnet }}_${{ matrix.cloud_env }}_${{ matrix.pkg_version }}_coverage.xml
- name: Upload Test Performance Report
uses: actions/upload-artifact@v3
with:
name: tests-performance
path: Snowflake.Data.Tests\windows_${{ matrix.dotnet }}_${{ matrix.cloud_env }}_performance.csv
path: Snowflake.Data.Tests\windows_${{ matrix.dotnet }}_${{ matrix.cloud_env }}_${{ matrix.pkg_version }}_performance.csv
- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v3
with:
# without the token code cov may fail because of Github limits https://github.com/codecov/codecov-action/issues/557
token: ${{ secrets.CODE_COV_UPLOAD_TOKEN }}
fail_ci_if_error: true
files: Snowflake.Data.Tests\windows_${{ matrix.dotnet }}_${{ matrix.cloud_env }}_coverage.xml
files: Snowflake.Data.Tests\windows_${{ matrix.dotnet }}_${{ matrix.cloud_env }}_${{ matrix.pkg_version }}_coverage.xml

test-linux:
name: Tests on Linux
Expand All @@ -95,6 +107,7 @@ jobs:
matrix:
dotnet: ['net6.0']
cloud_env: ['AZURE', 'GCP', 'AWS']
pkg_version: ['current', 'latest']
steps:
- uses: actions/checkout@v3
- name: Setup Dotnet
Expand All @@ -115,34 +128,46 @@ jobs:
run: |
gpg --quiet --batch --yes --decrypt --passphrase="$PARAMETER_SECRET" \
./.github/workflows/parameters/parameters_${{ matrix.cloud_env }}.json.gpg > Snowflake.Data.Tests/parameters.json
- name: Build Driver
- name: Build Driver Current
if: ${{ matrix.pkg_version == 'current' }}
run: |
cd Snowflake.Data.Tests
dotnet restore
dotnet build -f ${{ matrix.dotnet }}
- name: Build Driver Latest
if: ${{ matrix.pkg_version == 'latest' }}
run: |
cd Snowflake.Data.Tests
dotnet add ./../Snowflake.Data/Snowflake.Data.csproj package AWSSDK.S3
dotnet add ./../Snowflake.Data/Snowflake.Data.csproj package Google.Cloud.Storage.V1
dotnet add ./../Snowflake.Data/Snowflake.Data.csproj package Azure.Storage.Blobs
dotnet restore
dotnet build
dotnet build -f ${{ matrix.dotnet }}
- name: Run Tests
run: |
cd Snowflake.Data.Tests
dotnet-coverage collect "dotnet test --framework ${{ matrix.dotnet }} --no-build -l console;verbosity=normal" --output linux_${{ matrix.dotnet }}_${{ matrix.cloud_env }}_coverage.xml --output-format cobertura --settings coverage.config
dotnet-coverage collect "dotnet test --framework ${{ matrix.dotnet }} --no-build -l console;verbosity=normal" --output linux_${{ matrix.dotnet }}_${{ matrix.cloud_env }}_${{ matrix.pkg_version }}_coverage.xml --output-format cobertura --settings coverage.config
env:
snowflake_cloud_env: ${{ matrix.cloud_env }}
pkg_version: ${{ matrix.pkg_version }}
net_version: ${{ matrix.dotnet }}
- name: Upload Code Coverage Report
uses: actions/upload-artifact@v3
with:
name: code-coverage-report
path: Snowflake.Data.Tests/linux_${{ matrix.dotnet }}_${{ matrix.cloud_env }}_coverage.xml
path: Snowflake.Data.Tests/linux_${{ matrix.dotnet }}_${{ matrix.cloud_env }}_${{ matrix.pkg_version }}_coverage.xml
- name: Upload Test Performance Report
uses: actions/upload-artifact@v3
with:
name: tests-performance
path: Snowflake.Data.Tests/linux_${{ matrix.dotnet }}_${{ matrix.cloud_env }}_performance.csv
path: Snowflake.Data.Tests/linux_${{ matrix.dotnet }}_${{ matrix.cloud_env }}_${{ matrix.pkg_version }}_performance.csv
- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v3
with:
# without the token code cov may fail because of Github limits https://github.com/codecov/codecov-action/issues/557
token: ${{ secrets.CODE_COV_UPLOAD_TOKEN }}
fail_ci_if_error: true
files: Snowflake.Data.Tests/linux_${{ matrix.dotnet }}_${{ matrix.cloud_env }}_coverage.xml
files: Snowflake.Data.Tests/linux_${{ matrix.dotnet }}_${{ matrix.cloud_env }}_${{ matrix.pkg_version }}_coverage.xml

test-mac:
name: Tests on MAC
Expand All @@ -152,6 +177,7 @@ jobs:
matrix:
dotnet: ['net6.0']
cloud_env: ['AZURE', 'GCP', 'AWS']
pkg_version: ['current', 'latest']
steps:
- uses: actions/checkout@v3
- name: Setup Dotnet
Expand All @@ -172,31 +198,43 @@ jobs:
run: |
gpg --quiet --batch --yes --decrypt --passphrase="$PARAMETER_SECRET" \
./.github/workflows/parameters/parameters_${{ matrix.cloud_env }}.json.gpg > Snowflake.Data.Tests/parameters.json
- name: Build Driver
- name: Build Driver Current
if: ${{ matrix.pkg_version == 'current' }}
run: |
cd Snowflake.Data.Tests
dotnet restore
dotnet build -f ${{ matrix.dotnet }}
- name: Build Driver Latest
if: ${{ matrix.pkg_version == 'latest' }}
run: |
cd Snowflake.Data.Tests
dotnet add ./../Snowflake.Data/Snowflake.Data.csproj package AWSSDK.S3
dotnet add ./../Snowflake.Data/Snowflake.Data.csproj package Google.Cloud.Storage.V1
dotnet add ./../Snowflake.Data/Snowflake.Data.csproj package Azure.Storage.Blobs
dotnet restore
dotnet build
dotnet build -f ${{ matrix.dotnet }}
- name: Run Tests
run: |
cd Snowflake.Data.Tests
dotnet-coverage collect "dotnet test --framework ${{ matrix.dotnet }} --no-build -l console;verbosity=normal" --output macos_${{ matrix.dotnet }}_${{ matrix.cloud_env }}_coverage.xml --output-format cobertura --settings coverage.config
dotnet-coverage collect "dotnet test --framework ${{ matrix.dotnet }} --no-build -l console;verbosity=normal" --output macos_${{ matrix.dotnet }}_${{ matrix.cloud_env }}_${{ matrix.pkg_version }}_coverage.xml --output-format cobertura --settings coverage.config
env:
snowflake_cloud_env: ${{ matrix.cloud_env }}
pkg_version: ${{ matrix.pkg_version }}
net_version: ${{ matrix.dotnet }}
- name: Upload Code Coverage Report
uses: actions/upload-artifact@v3
with:
name: code-coverage-report
path: Snowflake.Data.Tests/macos_${{ matrix.dotnet }}_${{ matrix.cloud_env }}_coverage.xml
path: Snowflake.Data.Tests/macos_${{ matrix.dotnet }}_${{ matrix.cloud_env }}_${{ matrix.pkg_version }}_coverage.xml
- name: Upload Test Performance Report
uses: actions/upload-artifact@v3
with:
name: tests-performance
path: Snowflake.Data.Tests/macos_${{ matrix.dotnet }}_${{ matrix.cloud_env }}_performance.csv
path: Snowflake.Data.Tests/macos_${{ matrix.dotnet }}_${{ matrix.cloud_env }}_${{ matrix.pkg_version }}_performance.csv
- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v3
with:
# without the token code cov may fail because of Github limits https://github.com/codecov/codecov-action/issues/557
token: ${{ secrets.CODE_COV_UPLOAD_TOKEN }}
fail_ci_if_error: true
files: Snowflake.Data.Tests/macos_${{ matrix.dotnet }}_${{ matrix.cloud_env }}_coverage.xml
files: Snowflake.Data.Tests/macos_${{ matrix.dotnet }}_${{ matrix.cloud_env }}_${{ matrix.pkg_version }}_coverage.xml
3 changes: 2 additions & 1 deletion Snowflake.Data.Tests/SFBaseTest.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -246,12 +246,13 @@ public void CreateTestTimeArtifact()

var dotnetVersion = Environment.GetEnvironmentVariable("net_version");
var cloudEnv = Environment.GetEnvironmentVariable("snowflake_cloud_env");
var pkgVersion = Environment.GetEnvironmentVariable("pkg_version");

var separator = Path.DirectorySeparatorChar;

// We have to go up 3 times as the working directory path looks as follows:
// Snowflake.Data.Tests/bin/debug/{.net_version}/
File.WriteAllText($"..{separator}..{separator}..{separator}{GetOs()}_{dotnetVersion}_{cloudEnv}_performance.csv", resultText);
File.WriteAllText($"..{separator}..{separator}..{separator}{GetOs()}_{dotnetVersion}_{cloudEnv}_{pkgVersion}_performance.csv", resultText);
}

private static string s_connectionString => string.Format(ConnectionStringFmt,
Expand Down
1 change: 0 additions & 1 deletion Snowflake.Data/Snowflake.Data.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@
<PackageReference Include="AWSSDK.S3" Version="3.7.0.4" />
<PackageReference Include="Google.Cloud.Storage.V1" Version="4.6.0" />
<PackageReference Include="Azure.Storage.Blobs" Version="12.13.0" />
<PackageReference Include="Azure.Storage.Common" Version="12.12.0" />
<PackageReference Include="Mono.Unix" Version="7.1.0-final.1.21458.1" />
<PackageReference Include="Newtonsoft.Json" Version="13.0.3" />
<PackageReference Include="log4net" Version="2.0.12" />
Expand Down