Skip to content

Dev #162

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
henryborchers merged 11 commits into from
Aug 12, 2025
Merged

Dev #162

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
6b131ef
ci: Bump Windows10SDK installed in Windows Dockerfile version 18362 -…
henryborchers Aug 11, 2025
ac4073e
ci: installing packages from chocolatey is done in a single layer
henryborchers Aug 11, 2025
43baede
ci: GitHub actions use conan 2 api
henryborchers Aug 12, 2025
1d0b1e5
ci: GitHub actions use latest windows
henryborchers Aug 12, 2025
99e65dc
ci: GitHub actions fix upload-artifact name
henryborchers Aug 12, 2025
6be9eb0
ci: GitHub actions fix conan cache
henryborchers Aug 12, 2025
132cd62
ci: GitHub actions simplified
henryborchers Aug 12, 2025
4e61dbe
ci: GitHub actions no longer remove strawberry perl
henryborchers Aug 12, 2025
10ca0f3
ci: GitHub Actions update for building wheels
henryborchers Aug 12, 2025
523d818
ci: Jenkins times out in tox stage
henryborchers Aug 12, 2025
9571aee
ci: use manylinux_2_28 with cibuildwheel
henryborchers Aug 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 13 additions & 35 deletions .github/workflows/tox_matrix.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,19 +15,11 @@ jobs:
env:
PIP_EXTRA_INDEX_URL: ${{vars.PIP_EXTRA_INDEX_URL}}
UV_EXTRA_INDEX_URL: ${{vars.PIP_EXTRA_INDEX_URL}}
UV_CONSTRAINT: requirements-dev.txt
strategy:
matrix:
os: [ubuntu-latest, macos-latest, windows-2019]
os: [ubuntu-latest, macos-latest, windows-latest]
python-version: ['3.9', '3.10', '3.11', '3.12', '3.13']
include:
- os: macos-latest
compiler_version: 14
compiler_libcxx: libc++
- os: ubuntu-latest
compiler_version: 11.4
compiler_libcxx: libstdc++11
- os: windows-2019
compiler_version: 16
fail-fast: false
name: Python ${{ matrix.python-version }} ${{ matrix.os }} build

Expand All @@ -37,7 +29,7 @@ jobs:
with:
result-encoding: string
script: |
if ('${{matrix.os}}' === 'windows-2019'){
if ('${{matrix.os}}' === 'windows-latest'){
return 'C:/Users/runneradmin'
}
if ('${{matrix.os}}' === 'ubuntu-latest'){
Expand Down Expand Up @@ -68,24 +60,12 @@ jobs:
requirements/requirements-*.txt
- name: Install dependencies
run: |
pip install -r requirements-dev.txt
- name: Ensure conan settings has current compiler as valid
if: "!contains(matrix.os, 'windows')"
env:
CONAN_USER_HOME: "${{steps.conan-path-script.outputs.result}}"
run: |
conan config init
python ci/docker/linux/shared/ensure_compiler_support_by_conan.py $(conan config home)/settings.yml $(cc -dumpfullversion -dumpversion | grep -oE "^([0-9]+(\.)?)([0-9]+?)")
conan profile update settings.compiler.version=${{matrix.compiler_version}} default

- name: Remove Strawberry Perl (Windows only)
if: contains(matrix.os, 'windows')
run: Remove-Item –path C:\\Strawberry –recurse –force
pip install uv
- name: Cache Conan
id: cache-conan
uses: actions/cache@v3
with:
path: "${{steps.conan-path-script.outputs.result}}/.conan"
path: "${{steps.conan-path-script.outputs.result}}/.conan2"
key: ${{ runner.os }}-conan-${{ hashFiles('conanfile.py') }}

- name: Prebuild Conan packages (Windows)
Expand All @@ -95,39 +75,37 @@ jobs:
env:
CONAN_USER_HOME: "${{steps.conan-path-script.outputs.result}}"
run: |
call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Auxiliary\Build\vcvars64.bat"
conan profile new default --detect
conan install . --build missing --no-import
"C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\Tools\VsDevCmd.bat" -arch=amd64 && uvx --with cmake conan profile detect --exist-ok
"C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\Tools\VsDevCmd.bat" -arch=amd64 && uvx --with cmake conan install conanfile.py --build missing -of %TEMP%\ciwheelbuilder -pr:b=default

- name: Prebuild Conan packages
env:
CONAN_USER_HOME: "${{steps.conan-path-script.outputs.result}}"
if: "!contains(matrix.os, 'windows') && steps.cache-conan.outputs.cache-hit != 'true'"
run: |
conan install . --build missing --no-import
uvx --with cmake conan profile detect --exist-ok
uvx --with cmake conan install conanfile.py --build missing -of /tmp/ciwheelbuilder -pr:b=default


- name: Run Tox (Windows)
if: contains(matrix.os, 'windows')
shell: cmd
run: |
call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Auxiliary\Build\vcvars64.bat"
tox run
"C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\Tools\VsDevCmd.bat" -arch=amd64 && uvx --with tox-uv tox run
env:
TOX_ENV: "${{ steps.tox-env.outputs.result }}"
CONAN_USER_HOME: "${{steps.conan-path-script.outputs.result}}"
- name: Run Tox
if: "!contains(matrix.os, 'windows')"
run: cc --version && cc -dumpfullversion -dumpversion && tox run -vvv
run: cc --version && cc -dumpfullversion -dumpversion && uvx --with tox-uv tox run -vvv
env:
TOX_ENV: "${{ steps.tox-env.outputs.result }}"
CONAN_USER_HOME: "${{steps.conan-path-script.outputs.result}}"
CONAN_COMPILER_VERSION: ${{ matrix.compiler_version }}
CONAN_COMPILER_LIBCXX: ${{ matrix.compiler_libcxx }}

# CIBW_SOME_OPTION: value
- uses: actions/upload-artifact@v4
if: ${{ inputs.save }}
with:
name: uiucprescon.ocr-wheels
name: uiucprescon.imagevalidate-wheels
path: ./wheelhouse/*.whl

21 changes: 5 additions & 16 deletions .github/workflows/wheels.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,38 +9,27 @@ jobs:
env:
PIP_EXTRA_INDEX_URL: ${{vars.PIP_EXTRA_INDEX_URL}}
UV_EXTRA_INDEX_URL: ${{vars.PIP_EXTRA_INDEX_URL}}
UV_CONSTRAINT: "requirements/requirements-cibuildwheel.txt"
strategy:
matrix:
os: [
ubuntu-20.04,
windows-2019, macos-13]
include:
- os: macos-13
compiler_version: 13
- os: ubuntu-20.04
compiler_version: 10.2
- os: windows-2019
compiler_version: 16
ubuntu-latest, windows-latest, macos-latest]

steps:
- uses: actions/checkout@v3

# Used to host cibuildwheel
- uses: actions/setup-python@v3
- name: Install cibuildwheel
run: python -m pip install -r requirements/requirements-cibuildwheel.txt
run: python -m pip install uv

- name: Build wheels
if: contains(matrix.os, 'windows')
shell: cmd
run: |
call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Auxiliary\Build\vcvars64.bat"
python -m cibuildwheel --output-dir wheelhouse
"C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\Tools\VsDevCmd.bat" -arch=amd64 && uvx cibuildwheel --output-dir wheelhouse
env:
CONAN_COMPILER_VERSION: ${{ matrix.compiler_version }}
- name: Build wheels
if: "!contains(matrix.os, 'windows')"
run: python -m cibuildwheel --output-dir wheelhouse
env:
CONAN_COMPILER_VERSION: ${{ matrix.compiler_version }}
# CIBW_SOME_OPTION: value
run: uvx cibuildwheel --output-dir wheelhouse
86 changes: 49 additions & 37 deletions Jenkinsfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -851,13 +851,15 @@ pipeline {
node('docker && linux'){
try{
checkout scm
docker.image('python').inside{
sh(script: 'python3 -m venv venv && venv/bin/pip install --disable-pip-version-check uv')
envs = sh(
label: 'Get tox environments',
script: './venv/bin/uvx --quiet --constraint requirements-dev.txt --with tox-uv tox list -d --no-desc',
returnStdout: true,
).trim().split('\n')
timeout(10){
docker.image('python').inside{
sh(script: 'python3 -m venv venv && venv/bin/pip install --disable-pip-version-check uv')
envs = sh(
label: 'Get tox environments',
script: './venv/bin/uvx --quiet --constraint requirements-dev.txt --with tox-uv tox list -d --no-desc',
returnStdout: true,
).trim().split('\n')
}
}
} finally{
sh "${tool(name: 'Default', type: 'git')} clean -dfx"
Expand All @@ -873,24 +875,28 @@ pipeline {
node('docker && linux'){
def image
checkout scm
lock("${env.JOB_NAME} - ${env.NODE_NAME}"){
image = docker.build(UUID.randomUUID().toString(), '-f ci/docker/linux/tox/Dockerfile --build-arg PIP_EXTRA_INDEX_URL --build-arg PIP_INDEX_URL --build-arg UV_EXTRA_INDEX_URL --build-arg UV_INDEX_URL --build-arg PIP_DOWNLOAD_CACHE=/.cache/pip --build-arg UV_CACHE_DIR=/.cache/uv --build-arg CONAN_CENTER_PROXY_V1_URL .')
timeout(60){
lock("${env.JOB_NAME} - ${env.NODE_NAME}"){
image = docker.build(UUID.randomUUID().toString(), '-f ci/docker/linux/tox/Dockerfile --build-arg PIP_EXTRA_INDEX_URL --build-arg PIP_INDEX_URL --build-arg UV_EXTRA_INDEX_URL --build-arg UV_INDEX_URL --build-arg PIP_DOWNLOAD_CACHE=/.cache/pip --build-arg UV_CACHE_DIR=/.cache/uv --build-arg CONAN_CENTER_PROXY_V1_URL .')
}
}
try{
image.inside{
try{
sh( label: 'Running Tox',
script: """python3 -m venv /tmp/venv && /tmp/venv/bin/pip install --disable-pip-version-check uv
. /tmp/venv/bin/activate
uvx -p ${version} --constraint requirements-dev.txt --with tox-uv tox run -e ${toxEnv} -vvv
"""
)
} catch(e) {
sh(script: '''. ./venv/bin/activate
uv python list
'''
timeout(30){
image.inside{
try{
sh( label: 'Running Tox',
script: """python3 -m venv /tmp/venv && /tmp/venv/bin/pip install --disable-pip-version-check uv
. /tmp/venv/bin/activate
uvx -p ${version} --constraint requirements-dev.txt --with tox-uv tox run -e ${toxEnv} -vvv
"""
)
throw e
} catch(e) {
sh(script: '''. ./venv/bin/activate
uv python list
'''
)
throw e
}
}
}
} finally {
Expand Down Expand Up @@ -923,13 +929,15 @@ pipeline {
node('docker && windows'){
try{
checkout scm
docker.image(env.DEFAULT_PYTHON_DOCKER_IMAGE ? env.DEFAULT_PYTHON_DOCKER_IMAGE: 'python').inside("--mount source=${SHARED_PIP_CACHE_VOLUME_NAME},target=${env:PIP_CACHE_DIR} --mount source=uv_python_install_dir,target=${env.UV_PYTHON_INSTALL_DIR}"){
bat(script: 'python -m venv venv && venv\\Scripts\\pip install --disable-pip-version-check uv')
envs = bat(
label: 'Get tox environments',
script: '@.\\venv\\Scripts\\uvx --quiet --constraint requirements-dev.txt --with tox-uv tox list -d --no-desc',
returnStdout: true,
).trim().split('\r\n')
timeout(10){
docker.image(env.DEFAULT_PYTHON_DOCKER_IMAGE ? env.DEFAULT_PYTHON_DOCKER_IMAGE: 'python').inside("--mount source=${SHARED_PIP_CACHE_VOLUME_NAME},target=${env:PIP_CACHE_DIR} --mount source=uv_python_install_dir,target=${env.UV_PYTHON_INSTALL_DIR}"){
bat(script: 'python -m venv venv && venv\\Scripts\\pip install --disable-pip-version-check uv')
envs = bat(
label: 'Get tox environments',
script: '@.\\venv\\Scripts\\uvx --quiet --constraint requirements-dev.txt --with tox-uv tox list -d --no-desc',
returnStdout: true,
).trim().split('\r\n')
}
}
} finally{
bat "${tool(name: 'Default', type: 'git')} clean -dfx"
Expand All @@ -945,18 +953,22 @@ pipeline {
node('docker && windows'){
def image
checkout scm
lock("${env.JOB_NAME} - ${env.NODE_NAME}"){
image = docker.build(UUID.randomUUID().toString(), '-f scripts/resources/windows/Dockerfile --build-arg PIP_EXTRA_INDEX_URL --build-arg PIP_INDEX_URL --build-arg CHOCOLATEY_SOURCE --build-arg chocolateyVersion --build-arg PIP_DOWNLOAD_CACHE=c:/users/ContainerUser/appdata/local/pip --build-arg UV_INDEX_URL --build-arg UV_EXTRA_INDEX_URL --build-arg UV_CACHE_DIR=c:/users/ContainerUser/appdata/local/uv' + (env.DEFAULT_DOCKER_DOTNET_SDK_BASE_IMAGE ? " --build-arg CONAN_CENTER_PROXY_V1_URL --build-arg FROM_IMAGE=${env.DEFAULT_DOCKER_DOTNET_SDK_BASE_IMAGE} ": ' ') + '.')
timeout(60){
lock("${env.JOB_NAME} - ${env.NODE_NAME}"){
image = docker.build(UUID.randomUUID().toString(), '-f scripts/resources/windows/Dockerfile --build-arg PIP_EXTRA_INDEX_URL --build-arg PIP_INDEX_URL --build-arg CHOCOLATEY_SOURCE --build-arg chocolateyVersion --build-arg PIP_DOWNLOAD_CACHE=c:/users/ContainerUser/appdata/local/pip --build-arg UV_INDEX_URL --build-arg UV_EXTRA_INDEX_URL --build-arg UV_CACHE_DIR=c:/users/ContainerUser/appdata/local/uv' + (env.DEFAULT_DOCKER_DOTNET_SDK_BASE_IMAGE ? " --build-arg CONAN_CENTER_PROXY_V1_URL --build-arg FROM_IMAGE=${env.DEFAULT_DOCKER_DOTNET_SDK_BASE_IMAGE} ": ' ') + '.')
}
}
try{
checkout scm
try{
image.inside("--mount source=${SHARED_PIP_CACHE_VOLUME_NAME},target=${env:PIP_CACHE_DIR} --mount source=uv_python_install_dir,target=${env.UV_PYTHON_INSTALL_DIR}"){
powershell(label: 'Running Tox',
script: """uv python install cpython-${version}
uvx -p ${version} --constraint requirements-dev.txt --with tox-uv tox run -e ${toxEnv}
"""
)
timeout(30){
image.inside("--mount source=${SHARED_PIP_CACHE_VOLUME_NAME},target=${env:PIP_CACHE_DIR} --mount source=uv_python_install_dir,target=${env.UV_PYTHON_INSTALL_DIR}"){
powershell(label: 'Running Tox',
script: """uv python install cpython-${version}
uvx -p ${version} --constraint requirements-dev.txt --with tox-uv tox run -e ${toxEnv}
"""
)
}
}
} finally{
bat "${tool(name: 'Default', type: 'git')} clean -dfx"
Expand Down
Loading
Loading