Skip to content

ci : add ccache action to windows-cublas job #2893

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Mar 19, 2025
Merged

ci : add ccache action to windows-cublas job #2893

Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
940f6e8
ci : add ccache action to windows-cublas job
danbev Mar 17, 2025
4e84e1f
ci : add nvcc compilation cache to windows-cublas
danbev Mar 18, 2025
00edb1d
ci : add sccache compiler launcher for windows-cublas-ci
danbev Mar 18, 2025
64eeff6
ci : fix CUDA_CACHE_PATH for Windows
danbev Mar 18, 2025
5d812a4
ci : move cuda-toolkit installation before configuring sccache
danbev Mar 18, 2025
50adc81
ci : remove Jimver/cuda-toolkit and manually install CUDA
danbev Mar 18, 2025
b3e8aef
ci : set CUDACXX to use sccache for nvcc.exe
danbev Mar 18, 2025
9fa77cc
ci : add call to vcvars64.bat in build.yml
danbev Mar 18, 2025
a67883e
ci : copy vs cuda build customizations
danbev Mar 18, 2025
81b28a7
ci : remove mkdir command for MSBuildCustomizations
danbev Mar 18, 2025
d55cce8
ci : try to fix path to MSBuildExtensions
danbev Mar 18, 2025
b5b368e
ci : add copy of MSBuildExtensions to v12.2.0
danbev Mar 18, 2025
8b39d3e
ci : add id to cuda-toolkit jobs
danbev Mar 18, 2025
1dfd18b
ci : use env variable instead of step id
danbev Mar 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 93 additions & 8 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -590,14 +590,87 @@ jobs:
- name: Clone repository
uses: actions/checkout@v4

- name: Add msbuild to PATH
uses: microsoft/setup-msbuild@v2
- name: Install ccache
uses: hendrikmuhs/[email protected]
with:
key: ${{ github.job }}-${{ matrix.cuda-toolkit }}-${{ matrix.build }}
variant: sccache
evict-old-files: 1d

- name: Install CUDA Toolkit
id: cuda-toolkit
uses: Jimver/[email protected]
- name: Configure CUDA compilation cache
run: |
New-Item -Path "${{ github.workspace }}\cuda_cache" -ItemType Directory -Force
echo "CUDA_CACHE_PATH=${{ github.workspace }}\cuda_cache" | Out-File -FilePath $env:GITHUB_ENV -Append
echo "CUDA_CACHE_MAXSIZE=4294967296" | Out-File -FilePath $env:GITHUB_ENV -Append

- name: Install Cuda Toolkit 11.8.0
if: ${{ matrix.cuda-toolkit == '11.8.0' }}
run: |
mkdir -p "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8.0"
choco install unzip -y
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_cudart/windows-x86_64/cuda_cudart-windows-x86_64-11.8.89-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_nvcc/windows-x86_64/cuda_nvcc-windows-x86_64-11.8.89-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_nvrtc/windows-x86_64/cuda_nvrtc-windows-x86_64-11.8.89-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/libcublas/windows-x86_64/libcublas-windows-x86_64-11.8.1.74-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_nvtx/windows-x86_64/cuda_nvtx-windows-x86_64-11.8.86-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/visual_studio_integration/windows-x86_64/visual_studio_integration-windows-x86_64-11.8.86-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_nvprof/windows-x86_64/cuda_nvprof-windows-x86_64-11.8.87-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_cccl/windows-x86_64/cuda_cccl-windows-x86_64-11.8.89-archive.zip"
unzip '*.zip' -d "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8.0"
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8.0\cuda_cudart-windows-x86_64-11.8.89-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8.0" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8.0\cuda_nvcc-windows-x86_64-11.8.89-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8.0" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8.0\cuda_nvrtc-windows-x86_64-11.8.89-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8.0" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8.0\libcublas-windows-x86_64-11.8.1.74-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8.0" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8.0\cuda_nvtx-windows-x86_64-11.8.86-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8.0" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8.0\visual_studio_integration-windows-x86_64-11.8.86-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8.0" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8.0\cuda_nvprof-windows-x86_64-11.8.87-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8.0" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8.0\cuda_cccl-windows-x86_64-11.8.89-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8.0" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8.0\visual_studio_integration-windows-x86_64-11.8.86-archive\visual_studio_integration\MSBuildExtensions\*" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Microsoft\VC\v160\BuildCustomizations" /E /I /H /Y
echo "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8.0\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
echo "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8.0\libnvvp" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
echo "CUDA_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8.0" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
echo "CUDA_PATH_V11_8=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8.0" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8

- name: Install Cuda Toolkit 12.2
if: ${{ matrix.cuda-toolkit == '12.2.0' }}
run: |
mkdir -p "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2.0"
choco install unzip -y
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_cudart/windows-x86_64/cuda_cudart-windows-x86_64-12.2.140-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_nvcc/windows-x86_64/cuda_nvcc-windows-x86_64-12.2.140-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_nvrtc/windows-x86_64/cuda_nvrtc-windows-x86_64-12.2.140-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/libcublas/windows-x86_64/libcublas-windows-x86_64-12.2.5.6-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_nvtx/windows-x86_64/cuda_nvtx-windows-x86_64-12.2.140-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_profiler_api/windows-x86_64/cuda_profiler_api-windows-x86_64-12.2.140-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/visual_studio_integration/windows-x86_64/visual_studio_integration-windows-x86_64-12.2.140-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_nvprof/windows-x86_64/cuda_nvprof-windows-x86_64-12.2.142-archive.zip"
curl -O "https://developer.download.nvidia.com/compute/cuda/redist/cuda_cccl/windows-x86_64/cuda_cccl-windows-x86_64-12.2.140-archive.zip"
unzip '*.zip' -d "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2.0"
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2.0\cuda_cudart-windows-x86_64-12.2.140-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2.0" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2.0\cuda_nvcc-windows-x86_64-12.2.140-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2.0" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2.0\cuda_nvrtc-windows-x86_64-12.2.140-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2.0" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2.0\libcublas-windows-x86_64-12.2.5.6-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2.0" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2.0\cuda_nvtx-windows-x86_64-12.2.140-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2.0" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2.0\cuda_profiler_api-windows-x86_64-12.2.140-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2.0" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2.0\visual_studio_integration-windows-x86_64-12.2.140-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2.0" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2.0\cuda_nvprof-windows-x86_64-12.2.142-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2.0" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2.0\cuda_cccl-windows-x86_64-12.2.140-archive\*" "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2.0" /E /I /H /Y
xcopy "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2.0\visual_studio_integration-windows-x86_64-12.2.140-archive\visual_studio_integration\MSBuildExtensions\*" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Microsoft\VC\v160\BuildCustomizations" /E /I /H /Y
echo "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2.0\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
echo "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2.0\libnvvp" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
echo "CUDA_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2.0" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
echo "CUDA_PATH_V12_2=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2.0" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8

- name: Cache CUDA compilation results
uses: actions/cache@v3
with:
cuda: '${{ matrix.cuda-toolkit }}'
path: ${{ github.workspace }}/cuda_cache
key: cuda-cache-${{ runner.os }}-${{ matrix.cuda-toolkit }}-${{ hashFiles('**/*.cu', '**/*.cuh') }}
restore-keys: |
cuda-cache-${{ runner.os }}-${{ matrix.cuda-toolkit }}-

- name: Add msbuild to PATH
uses: microsoft/setup-msbuild@v2

- name: Install 7-Zip
run: choco install 7zip -y
Expand All @@ -613,22 +686,34 @@ jobs:
- name: Configure CMake
shell: cmd
run: |
call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Auxiliary\Build\vcvars64.bat"
set "CUDACXX=sccache %CUDA_PATH%\bin\nvcc.exe"
cmake -S . -B ./build -A ${{ matrix.arch }} ^
-DCMAKE_BUILD_TYPE=${{ matrix.build }} ^
-DGGML_CUDA=${{ matrix.cublas }} ^
-DCMAKE_CUDA_ARCHITECTURES=all ^
-DWHISPER_SDL2=${{ matrix.sdl2 }} ^
-DCMAKE_CUDA_COMPILER_LAUNCHER=sccache ^
-DCMAKE_C_COMPILER_LAUNCHER=sccache ^
-DCMAKE_CXX_COMPILER_LAUNCHER=sccache ^
-DCMAKE_MSVC_DEBUG_INFORMATION_FORMAT=Embedded ^
-DCMAKE_POLICY_CMP0141=NEW ^
-DSDL2_DIR="%SDL2_DIR%"

- name: Check sccache status after build
run: |
sccache --show-stats

- name: Build Project
shell: cmd
run: |
call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Auxiliary\Build\vcvars64.bat"
cd ./build
cmake --build . --config ${{ matrix.build }}
cmake --build . --config ${{ matrix.build }} -j %NUMBER_OF_PROCESSORS%

- name: Copy CUDA DLLs
run: |
Get-ChildItem "${{ steps.cuda-toolkit.outputs.CUDA_PATH }}/bin/" -Filter "*.dll" |
Get-ChildItem "$env:CUDA_PATH\bin\" -Filter "*.dll" |
Copy-Item -Destination "build/bin/${{ matrix.build }}"

- name: Copy SDL2.dll
Expand Down
Loading