Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
225 commits
Select commit Hold shift + click to select a range
a56601c
CI for 314/314t
paultiq Sep 10, 2025
dbb3748
Chore: CI - Disable everything but 314 for now.
paultiq Sep 10, 2025
22e3a13
bump uv version
paultiq Sep 10, 2025
e652913
cache too
paultiq Sep 10, 2025
64ab8f8
Chore: CI - reduce matrix and enable experimental for 314
paultiq Sep 10, 2025
e8ec78a
enable the right groups
paultiq Sep 10, 2025
b1a52c0
for now, make sure uv is using the right python version using python-…
paultiq Sep 10, 2025
0869d1c
For testing, let's cache the build dir, for a single target
paultiq Sep 10, 2025
af8c433
Simplify cache
paultiq Sep 10, 2025
780288b
Disable coverage
paultiq Sep 10, 2025
242ee31
Disable coverage
paultiq Sep 10, 2025
b112a71
chore: disable sdist and try to get ci happier
paultiq Sep 10, 2025
dac704a
add ccache
paultiq Sep 10, 2025
e3e3c44
explicitly set ccache dir
paultiq Sep 10, 2025
214d335
debug info: why failing
paultiq Sep 10, 2025
5ba7743
always save cache,
paultiq Sep 10, 2025
b5769ea
back to 314t
paultiq Sep 10, 2025
4c4413c
setting up ccache
paultiq Sep 10, 2025
45aab9c
ccache is already installed, just set env
paultiq Sep 10, 2025
177aa6f
install ccache
paultiq Sep 10, 2025
e42b3d0
fixing uv not using right version
paultiq Sep 10, 2025
3a954b6
Use setup-python
paultiq Sep 10, 2025
9d66ab1
Close quotes.
paultiq Sep 10, 2025
d0822a4
testing ccache
paultiq Sep 10, 2025
49974ac
reenable ccache
paultiq Sep 10, 2025
21235c4
cache lock free
paultiq Sep 10, 2025
ab98aa1
ccache
paultiq Sep 10, 2025
d033110
debugging project dir
paultiq Sep 10, 2025
8498e62
add debug info
paultiq Sep 10, 2025
17cd50f
make sure the cache dir is mounted
paultiq Sep 10, 2025
6095335
change cache key
paultiq Sep 10, 2025
4e5cfdb
try to force ccache
paultiq Sep 10, 2025
32caa8b
add test file, still figuring out ccache caching
paultiq Sep 10, 2025
04fdedd
test file
paultiq Sep 10, 2025
b2757fd
move into build dir
paultiq Sep 10, 2025
d259d19
simnplify
paultiq Sep 10, 2025
fae60ce
option 2: sccache
paultiq Sep 10, 2025
b144404
sccache
paultiq Sep 10, 2025
15361cf
try
paultiq Sep 10, 2025
89ad08c
increase build
paultiq Sep 10, 2025
e7b438a
pass env
paultiq Sep 10, 2025
27b4252
use v.0.10.0 sccache
paultiq Sep 10, 2025
d788479
curl not wget
paultiq Sep 10, 2025
8a9ee03
double sccache
paultiq Sep 10, 2025
d7de926
triple? Wrong direction.
paultiq Sep 10, 2025
11b0144
double the fun
paultiq Sep 10, 2025
eed25a6
duckdb already sets sccache
paultiq Sep 10, 2025
2bb9278
double => 1
paultiq Sep 10, 2025
8a761a9
test again
paultiq Sep 10, 2025
8efe563
Pass variables explicitly
paultiq Sep 10, 2025
f3c9431
pass env
paultiq Sep 10, 2025
2b65e52
export
paultiq Sep 10, 2025
2853060
verify vars
paultiq Sep 10, 2025
36dd8a2
fix conditional logic for 3.14
paultiq Sep 10, 2025
4df812d
debug logging
paultiq Sep 11, 2025
28556cd
true not on
paultiq Sep 11, 2025
6ed58ec
v2
paultiq Sep 11, 2025
4cad124
self hosted, test one file
paultiq Sep 11, 2025
aa7a974
self hosted
paultiq Sep 11, 2025
aba20d8
self hosted 2
paultiq Sep 11, 2025
f705a88
fix python version typo
paultiq Sep 11, 2025
97a0f50
start the server
paultiq Sep 11, 2025
777c9d8
a
paultiq Sep 11, 2025
7556632
add debug
paultiq Sep 11, 2025
753e59a
bump
paultiq Sep 11, 2025
f2c5a75
add debug
paultiq Sep 11, 2025
c50dc8f
install directly
paultiq Sep 11, 2025
e8c27b8
typo
paultiq Sep 11, 2025
4ded61b
test sccache
paultiq Sep 11, 2025
b0a9b10
fix test
paultiq Sep 11, 2025
eb4fd06
grant write perm for sccache
paultiq Sep 11, 2025
ea0f775
Add write perm
paultiq Sep 11, 2025
37eaf01
change concurrency
paultiq Sep 11, 2025
6772ea5
back to action
paultiq Sep 11, 2025
8337cc9
sccache
paultiq Sep 11, 2025
c718468
bump
paultiq Sep 11, 2025
4a17f55
bump2
paultiq Sep 11, 2025
f86ec1f
"ACTIONS_RESULTS_URL",
paultiq Sep 11, 2025
9aed4ad
set all
paultiq Sep 11, 2025
a37c597
upd
paultiq Sep 11, 2025
af6fa95
ACTIONS_RESULTS_URL
paultiq Sep 11, 2025
0edec7c
try a writeable dir
paultiq Sep 11, 2025
20f5146
pass dir through
paultiq Sep 11, 2025
fea57f9
capture logging
paultiq Sep 11, 2025
ded10ed
SCCACHE_START_SERVER=1
paultiq Sep 11, 2025
3d11b76
2
paultiq Sep 11, 2025
b789577
v2
paultiq Sep 11, 2025
37ad2e7
ACTIONS_CACHE_SERVICE_V2
paultiq Sep 11, 2025
df3b2b4
keep debugging
paultiq Sep 11, 2025
068fadd
disable concurrency
paultiq Sep 11, 2025
d4d6866
Disable cache
paultiq Sep 11, 2025
f84dd56
no sccache at top level
paultiq Sep 11, 2025
60abe5c
eanble for ubuntu
paultiq Sep 11, 2025
c5f1b6a
emit versions
paultiq Sep 11, 2025
8101d10
emit versions self hosted
paultiq Sep 11, 2025
175cac4
remove unneeded stuff
paultiq Sep 11, 2025
326e846
back to test
paultiq Sep 11, 2025
a449c73
make tests pass
paultiq Sep 11, 2025
a2a5cf8
use nightly pandas
paultiq Sep 11, 2025
59713ea
try 2
paultiq Sep 11, 2025
735f5af
move index
paultiq Sep 11, 2025
8af90c6
update indices
paultiq Sep 11, 2025
6c3c84d
accept new warning for pandas deprecation
paultiq Sep 11, 2025
0ac07f4
keep testing locally
paultiq Sep 11, 2025
f762785
on my own
paultiq Sep 11, 2025
03bb7e9
runs-on: self-hosted
paultiq Sep 11, 2025
0ff34f9
dont r
paultiq Sep 11, 2025
633e887
nope
paultiq Sep 11, 2025
2be590f
cache uv and exclude warnings in CIBW_TEST_COMMAND
paultiq Sep 11, 2025
f33bce4
use a uv cache dir
paultiq Sep 11, 2025
f93f112
quoting
paultiq Sep 11, 2025
bc925c7
Move filter earlier
paultiq Sep 11, 2025
1625615
reorder warnings
paultiq Sep 11, 2025
73953f9
pass exclusion
paultiq Sep 11, 2025
a6afd08
it's an exception
paultiq Sep 11, 2025
a60e533
Use the nightly wheel, don't rebuild
paultiq Sep 11, 2025
53ad916
set index
paultiq Sep 11, 2025
660ba3f
default for <3.14
paultiq Sep 11, 2025
223f03a
undo
paultiq Sep 11, 2025
d3acfb3
drop back a version
paultiq Sep 11, 2025
a64e156
populate cache
paultiq Sep 11, 2025
0231b7d
fix env
paultiq Sep 11, 2025
6a086cc
building quickly
paultiq Sep 11, 2025
e807a90
remove debug
paultiq Sep 11, 2025
bd0f564
fix sccache for the millionth time
paultiq Sep 11, 2025
73d99f2
put the warning
paultiq Sep 11, 2025
24d7013
avoid sccache error
paultiq Sep 11, 2025
75172d2
remove the linux entry
paultiq Sep 11, 2025
2691875
speed up windows
paultiq Sep 11, 2025
c1ba9bd
before-test
paultiq Sep 11, 2025
bf80e77
sccache
paultiq Sep 11, 2025
bc43b83
sccache logging
paultiq Sep 11, 2025
ffc6251
sccache stats
paultiq Sep 11, 2025
7b522a6
not one line
paultiq Sep 11, 2025
5b8fdb7
pass as cmake arg
paultiq Sep 11, 2025
017f19f
sccache
paultiq Sep 11, 2025
507e7ee
passing sccache
paultiq Sep 11, 2025
801606a
sccache
paultiq Sep 11, 2025
37fbed1
sccache before ccache
paultiq Sep 11, 2025
82af19c
add direct path
paultiq Sep 11, 2025
e1019ad
sccache
paultiq Sep 12, 2025
834790d
again
paultiq Sep 12, 2025
8666c2b
test
paultiq Sep 12, 2025
16e10fc
remove var, let win detect
paultiq Sep 12, 2025
0f3286b
debugging win
paultiq Sep 12, 2025
382b305
f2
paultiq Sep 12, 2025
f86fad2
fix linux
paultiq Sep 12, 2025
0fe2bf3
win4
paultiq Sep 12, 2025
2a15583
t3
paultiq Sep 12, 2025
8ac483a
install ninja
paultiq Sep 12, 2025
3a5654f
use ninja
paultiq Sep 12, 2025
56dc713
force nnnniiinja
paultiq Sep 12, 2025
1d14571
double sccache
paultiq Sep 12, 2025
d313986
Try mac, let's
paultiq Sep 12, 2025
7ec8c25
try to fix windows
paultiq Sep 12, 2025
a686e75
win3
paultiq Sep 12, 2025
0882af8
set cl for ninja
paultiq Sep 12, 2025
7215cd2
try
paultiq Sep 12, 2025
0e0a2d6
PATH
paultiq Sep 12, 2025
dc5705e
t6
paultiq Sep 12, 2025
f6b3152
quoting
paultiq Sep 12, 2025
0ad7251
toml
paultiq Sep 12, 2025
b4220ed
revert back to half working
paultiq Sep 12, 2025
3283319
just cache python
paultiq Sep 12, 2025
a6826eb
install cibuildwheel
paultiq Sep 12, 2025
fd66d35
the rest
paultiq Sep 12, 2025
b55429b
s3
paultiq Sep 12, 2025
09faf75
b
paultiq Sep 12, 2025
e3d50ce
pin setup-uv version
paultiq Sep 12, 2025
3760692
3.11
paultiq Sep 12, 2025
6c6bbea
no-sync
paultiq Sep 12, 2025
bacf97b
version
paultiq Sep 12, 2025
24f445d
zz
paultiq Sep 12, 2025
af42e97
zz
paultiq Sep 12, 2025
105d6e9
paths
paultiq Sep 12, 2025
20bb15d
try again
paultiq Sep 12, 2025
b11283f
dynamic
paultiq Sep 12, 2025
7f02915
z
paultiq Sep 12, 2025
05017ca
c
paultiq Sep 12, 2025
af83398
fix escapt
paultiq Sep 12, 2025
0253315
d
paultiq Sep 12, 2025
639f331
move to action
paultiq Sep 12, 2025
daa0e76
sd
paultiq Sep 12, 2025
1afe60e
t
paultiq Sep 12, 2025
021f6f2
deb
paultiq Sep 12, 2025
435e579
r
paultiq Sep 12, 2025
c5649b1
z
paultiq Sep 12, 2025
f4bae55
x
paultiq Sep 12, 2025
ed392e0
f
paultiq Sep 12, 2025
f6931a6
sd
paultiq Sep 12, 2025
db880f0
ad
paultiq Sep 12, 2025
1f33aa7
asd
paultiq Sep 12, 2025
30fefd3
blank
paultiq Sep 12, 2025
8ec7940
all
paultiq Sep 12, 2025
8d2eb55
win.. land the plane
paultiq Sep 12, 2025
5dbc94a
try again
paultiq Sep 12, 2025
ba735b6
comments
paultiq Sep 12, 2025
075ada9
t
paultiq Sep 12, 2025
573a83a
more debug
paultiq Sep 12, 2025
b9fae5c
trying to get uv to cache
paultiq Sep 12, 2025
4abfece
tak 2
paultiq Sep 12, 2025
6fa6786
-b
paultiq Sep 12, 2025
3bd0a61
no auto
paultiq Sep 12, 2025
532e751
set arch
paultiq Sep 12, 2025
775a0f7
machine
paultiq Sep 12, 2025
6716110
d
paultiq Sep 12, 2025
927d4fc
3
paultiq Sep 12, 2025
b658f46
faster
paultiq Sep 12, 2025
b2d689d
v
paultiq Sep 12, 2025
23b44bd
a
paultiq Sep 12, 2025
d4efd13
hardcode
paultiq Sep 12, 2025
2829646
vcvars
paultiq Sep 12, 2025
1cbe999
unity
paultiq Sep 12, 2025
6300496
linux spot
paultiq Sep 12, 2025
e37d057
debug
paultiq Sep 12, 2025
1bd13ae
windows
paultiq Sep 12, 2025
6b97a4c
right param
paultiq Sep 12, 2025
1e0ed9d
Flag
paultiq Sep 12, 2025
cc94766
strip down to repro
paultiq Sep 12, 2025
223b749
d
paultiq Sep 12, 2025
3a99e41
impl
paultiq Sep 12, 2025
46565d4
abi-flags
paultiq Sep 12, 2025
2ed610f
bigger matrix
paultiq Sep 12, 2025
4d04875
Run test in serial
paultiq Sep 12, 2025
bd4e5ee
Dont make things worse
paultiq Sep 12, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
File renamed without changes.
2 changes: 1 addition & 1 deletion .github/workflows/cleanup_pypi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ jobs:
- name: Install Astral UV
uses: astral-sh/setup-uv@v6
with:
version: "0.7.14"
version: "0.8.16"

- name: Run Cleanup
env:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/coverage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ jobs:
- name: Install Astral UV and enable the cache
uses: astral-sh/setup-uv@v6
with:
version: "0.7.14"
version: "0.8.16"
python-version: 3.9
enable-cache: true
cache-suffix: -${{ github.workflow }}
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/on_pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,9 @@ on:
- '!.github/workflows/on_push.yml'
- '!.github/workflows/coverage.yml'

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
# concurrency:
# group: ${{ github.workflow }}-${{ github.ref }}
# cancel-in-progress: true

jobs:
submodule_sanity_guard:
Expand Down
7 changes: 4 additions & 3 deletions .github/workflows/packaging.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,9 +52,9 @@ on:
required: false
type: string

concurrency:
group: packaging-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
# concurrency:
# group: packaging-${{ github.workflow }}-${{ github.ref }}
# cancel-in-progress: true

defaults:
run:
Expand All @@ -63,6 +63,7 @@ defaults:
jobs:
build_sdist:
name: Build an sdist and determine versions
if: ${{ github.event_name != 'pull_request' }}
uses: ./.github/workflows/packaging_sdist.yml
with:
testsuite: all
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/packaging_sdist.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ jobs:
- name: Install Astral UV
uses: astral-sh/setup-uv@v6
with:
version: "0.7.14"
version: "0.8.16"
python-version: 3.11

- name: Build sdist
Expand Down
76 changes: 61 additions & 15 deletions .github/workflows/packaging_wheels.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,12 @@ on:
jobs:
build_wheels:
name: 'Wheel: ${{ matrix.python }}-${{ matrix.platform.cibw_system }}_${{ matrix.platform.arch }}'
permissions:
actions: write # For GitHub Actions cache write access
strategy:
fail-fast: false
matrix:
python: [ cp39, cp310, cp311, cp312, cp313 ]
python: [ cp313, cp314t ] # cp39, cp310, cp311, cp312, cp313, cp314, cp314t
platform:
- { os: windows-2025, arch: amd64, cibw_system: win }
- { os: ubuntu-24.04, arch: x86_64, cibw_system: manylinux }
Expand All @@ -53,45 +55,89 @@ jobs:
uv export --only-group test --no-emit-project --output-file pylock.toml --directory {project} &&
uv pip install -r pylock.toml
CIBW_TEST_COMMAND: >
uv run -v pytest ${{ inputs.testsuite == 'fast' && './tests/fast' || './tests' }} --verbose --ignore=./tests/stubs

sccache --show-stats &&
uv run -v pytest ${{ inputs.testsuite == 'fast' && './tests/fast' || './tests' }} --verbose --ignore=./tests/stubs
SCCACHE_GHA_ENABLED: "on"
SCCACHE_C_CUSTOM_CACHE_BUSTER: ${{ toJSON(matrix) }}
ACTIONS_CACHE_SERVICE_V2: "1"
PYTHON_GIL: "1"
# Needed for 3.14t to indicate that it's OK to enable the GIL. Noop for other versions.
steps:
- name: Checkout DuckDB Python
uses: actions/checkout@v4
uses: actions/checkout@v5
with:
ref: ${{ inputs.duckdb-python-sha }}
fetch-depth: 0
submodules: true

# The host doesn't need sccache, only the cibuildwheel container
# - name: Setup sccache # idea from https://github.com/pypa/cibuildwheel/issues/1030
# uses: mozilla-actions/[email protected]
# with:
# version: "v0.10.0"
- name: Configure Cache Env
uses: actions/github-script@v7
with:
script: |
core.exportVariable('ACTIONS_RESULTS_URL', process.env.ACTIONS_RESULTS_URL || '');
core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || '');
- name: Checkout DuckDB
shell: bash
if: ${{ inputs.duckdb-sha }}
run: |
cd external/duckdb
git fetch origin
git checkout ${{ inputs.duckdb-sha }}

# Make sure that OVERRIDE_GIT_DESCRIBE is propagated to cibuildwhel's env, also when it's running linux builds
- name: Set OVERRIDE_GIT_DESCRIBE
if: ${{ inputs.set-version != '' }}
run: echo "CIBW_ENVIRONMENT=OVERRIDE_GIT_DESCRIBE=${{ inputs.set-version }}" >> $GITHUB_ENV

run: echo "CIBW_ENVIRONMENT=OVERRIDE_GIT_DESCRIBE=${{ inputs.set-version }} SKBUILD_BUILD_DIR=build/${{ matrix.python }}-${{ matrix.platform.arch }}" >> $GITHUB_ENV
# Install Astral UV, which will be used as build-frontend for cibuildwheel
- uses: astral-sh/setup-uv@v6
with:
version: "0.7.14"
enable-cache: false
version: "0.8.16"
enable-cache: true
activate-environment: true
cache-suffix: -${{ matrix.python }}-${{ matrix.platform.cibw_system }}_${{ matrix.platform.arch }}

- name: Build${{ inputs.testsuite != 'none' && ' and test ' || ' ' }}wheels
uses: pypa/[email protected]
python-version: ${{
fromJSON('{"cp39":"3.9","cp310":"3.10","cp311":"3.11","cp312":"3.12","cp313":"3.13","cp314":"3.14","cp314t":"3.14t"}')[matrix.python]
}}
- name: Install cibuildwheel
run: |
echo "Installed uv version is ${{ steps.setup-uv.outputs.uv-version }}"
uv pip install cibuildwheel==3.1.4
- name: Build${{ inputs.testsuite != 'none' && ' and test ' || ' ' }}wheels (Windows)
if: matrix.platform.cibw_system == 'win'
shell: cmd
run: |
for /f "usebackq tokens=*" %%i in (`"%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe" -latest -products * -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64 -property installationPath`) do (
echo Setting up x64 build environment...
call "%%i\VC\Auxiliary\Build\vcvars64.bat"
echo Verifying compiler paths:
where cl
echo VCINSTALLDIR=%VCINSTALLDIR%
echo Platform=%Platform%
uv run --no-sync python -m cibuildwheel --output-dir wheelhouse
)
env:
CIBW_ARCHS: ${{ matrix.platform.arch == 'amd64' && 'AMD64' || matrix.platform.arch }}
CIBW_BUILD: ${{ matrix.python }}-${{ matrix.platform.cibw_system }}_${{ matrix.platform.arch }}

CIBW_ENVIRONMENT_WINDOWS: SKBUILD_BUILD_DIR=build/${{ matrix.python }}-${{ matrix.platform.arch }} CMAKE_VERBOSE_MAKEFILE=ON
CIBW_BUILD_VERBOSITY: 3
- name: Build${{ inputs.testsuite != 'none' && ' and test ' || ' ' }}wheels (Non-Windows)
if: matrix.platform.cibw_system != 'win'
run: uv run --no-sync python -m cibuildwheel --output-dir wheelhouse
env:
CIBW_ARCHS: ${{ matrix.platform.arch == 'amd64' && 'AMD64' || matrix.platform.arch }}
CIBW_BUILD: ${{ matrix.python }}-${{ matrix.platform.cibw_system }}_${{ matrix.platform.arch }}
CIBW_ENVIRONMENT_LINUX: SKBUILD_BUILD_DIR=build/${{ matrix.python }}-${{ matrix.platform.arch }} CMAKE_C_COMPILER_LAUNCHER="" CMAKE_CXX_COMPILER_LAUNCHER=""
CIBW_ENVIRONMENT_MACOS: SKBUILD_BUILD_DIR=build/${{ matrix.python }}-${{ matrix.platform.arch }} CMAKE_C_COMPILER_LAUNCHER="" CMAKE_CXX_COMPILER_LAUNCHER=""
CIBW_BUILD_VERBOSITY: 3
CIBW_REPAIR_WHEEL_COMMAND_LINUX: |
sccache --show-stats
auditwheel repair -w {dest_dir} {wheel}
- name: Upload wheel
uses: actions/upload-artifact@v4
with:
name: wheel-${{ matrix.python }}-${{ matrix.platform.cibw_system }}_${{ matrix.platform.arch }}
path: wheelhouse/*.whl
compression-level: 0
compression-level: 0
142 changes: 127 additions & 15 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,10 @@ all = [ # users can install duckdb with 'duckdb[all]', which will install this l
"ipython", # used in duckdb.query_graph
"fsspec", # used in duckdb.filesystem
"numpy", # used in duckdb.experimental.spark and in duckdb.fetchnumpy()
"pandas", # used for pandas dataframes all over the place
"pyarrow", # used for pyarrow support
"adbc_driver_manager", # for the adbc driver (TODO: this should live under the duckdb package)
"pandas; python_version < '3.14'", # used for pandas dataframes all over the place
"pandas>=2.3.0.dev0; python_version >= '3.14'", # nightly build for Python 3.14
"pyarrow; python_version < '3.14'", # used for pyarrow support
"adbc_driver_manager; python_version < '3.14'", # for the adbc driver (TODO: this should live under the duckdb package)
]

######################################################################################################
Expand Down Expand Up @@ -204,6 +205,7 @@ required-environments = [ # ... but do always resolve for all of them
"python_version >= '3.9' and sys_platform == 'linux' and platform_machine == 'x86_64'",
"python_version >= '3.9' and sys_platform == 'linux' and platform_machine == 'aarch64'",
]
prerelease = "allow" # for 3.14

# We just need pytorch for tests, wihtout GPU acceleration. PyPI doesn't host a cpu-only version for Linux, so we have
# to configure the index url for cpu-only pytorch manually
Expand All @@ -212,6 +214,11 @@ name = "pytorch-cpu"
url = "https://download.pytorch.org/whl/cpu"
explicit = true

[[tool.uv.index]]
name = "scientific-python-nightly"
url = "https://pypi.anaconda.org/scientific-python-nightly-wheels/simple"
explicit = true

[tool.uv.sources]
torch = [ { index = "pytorch-cpu" } ]
torchvision = [ { index = "pytorch-cpu" } ]
Expand All @@ -227,23 +234,27 @@ test = [ # dependencies used for running tests
"pytest",
"pytest-reraise",
"pytest-timeout",
# "pytest-randomly", # Ensures non-deterministic test order
# "pytest-xdist", # Parallel tests (separate processes)
# "pytest-run-parallel", # For free-threading testing
"mypy",
"coverage",
"gcovr",
"gcovr; python_version < '3.14'",
"gcsfs",
"packaging",
"polars",
"polars; python_version < '3.14'",
"psutil",
"py4j",
"pyotp",
"pyspark",
"pyspark; python_version < '3.14'",
"pytz",
"requests",
"urllib3",
"fsspec>=2022.11.0",
"pandas>=2.0.0",
"pyarrow>=18.0.0",
"torch>=2.2.2; sys_platform != 'darwin' or platform_machine != 'x86_64' or python_version < '3.13'",
"pandas>=2.0.0; python_version < '3.14'",
"pandas>=2.3.0.dev0; python_version >= '3.14'",
"pyarrow>=18.0.0; python_version < '3.14'",
"torch>=2.2.2; python_version < '3.14' and (sys_platform != 'darwin' or platform_machine != 'x86_64' or python_version < '3.13')",
"tensorflow==2.14.0; sys_platform == 'darwin' and python_version < '3.12'",
"tensorflow-cpu>=2.14.0; sys_platform == 'linux' and platform_machine != 'aarch64' and python_version < '3.12'",
"tensorflow-cpu>=2.14.0; sys_platform == 'win32' and python_version < '3.12'",
Expand All @@ -256,10 +267,11 @@ scripts = [ # dependencies used for running scripts
"ipython",
"ipywidgets",
"numpy",
"pandas",
"pandas; python_version < '3.14'",
"pandas>=2.3.0.dev0; python_version >= '3.14'",
"pcpp",
"polars",
"pyarrow",
"polars; python_version < '3.14'",
"pyarrow; python_version < '3.14'",
"pytz"
]
pypi = [ # dependencies used by the pypi cleanup script
Expand Down Expand Up @@ -305,6 +317,7 @@ filterwarnings = [
# Pyspark is throwing these warnings
"ignore:distutils Version classes are deprecated:DeprecationWarning",
"ignore:is_datetime64tz_dtype is deprecated:DeprecationWarning",
"ignore:ChainedAssignmentError.*:FutureWarning"
]

[tool.coverage.run]
Expand Down Expand Up @@ -379,12 +392,111 @@ manylinux-x86_64-image = "manylinux_2_28"
manylinux-pypy_x86_64-image = "manylinux_2_28"
manylinux-aarch64-image = "manylinux_2_28"
manylinux-pypy_aarch64-image = "manylinux_2_28"
enable = ["cpython-freethreading", "cpython-prerelease"]

[tool.cibuildwheel.linux]
before-build = ["yum install -y ccache"]
environment-pass = ["SCCACHE_GHA_ENABLED", "ACTIONS_RUNTIME_TOKEN", "ACTIONS_RESULTS_URL", "ACTIONS_CACHE_SERVICE_V2", "SCCACHE_C_CUSTOM_CACHE_BUSTER", "PYTHON_GIL"]
before-build = [
"if [ \"$(uname -m)\" = \"aarch64\" ]; then ARCH=aarch64; else ARCH=x86_64; fi",
"curl -L https://github.com/mozilla/sccache/releases/download/v0.10.0/sccache-v0.10.0-${ARCH}-unknown-linux-musl.tar.gz | tar xz",
"cp sccache-v0.10.0-${ARCH}-unknown-linux-musl/sccache /usr/bin",
"which sccache",
"echo 'Compiler launcher detection order check:'",
"which ccache || echo 'ccache not found'",
"which sccache || echo 'sccache not found'",
"echo 'PATH priority test:'",
"for prog in ccache sccache; do echo \"$prog: $(which $prog 2>/dev/null || echo 'not found')\"; done",
"mkdir -p /root/.config/sccache && touch /root/.config/sccache/config",
"mkdir -p /tmp/sccache-cache",
"export SCCACHE_DIR=/tmp/sccache-cache",
"if [ -z \"$ACTIONS_RESULTS_URL\" ]; then echo \"ERROR: ACTIONS_RESULTS_URL not set in container\" && exit 1; fi",
"if [ -z \"$ACTIONS_RUNTIME_TOKEN\" ]; then echo \"ERROR: ACTIONS_RUNTIME_TOKEN not set in container\" && exit 1; fi",
"echo \"GitHub Actions environment variables verified in container\"",
"echo \"Testing sccache GitHub Actions cache access...\"",
"sccache --start-server",
"SCCACHE_LOG=debug sccache --show-stats 2>&1 | head -10 || true",
"echo 'int main(){return 0;}' > test.c",
"SCCACHE_LOG=debug sccache gcc -c test.c -o test.o 2>&1 | head -20 || true",
"sccache --show-stats",
"echo \"Sccache test complete\""
]
before-test = "sccache --show-stats"

[tool.cibuildwheel.macos]
before-build = ["brew install ccache"]
environment-pass = ["SCCACHE_GHA_ENABLED", "ACTIONS_RUNTIME_TOKEN", "ACTIONS_RESULTS_URL", "ACTIONS_CACHE_SERVICE_V2", "SCCACHE_C_CUSTOM_CACHE_BUSTER", "PYTHON_GIL"]
before-build = [
"brew install sccache",
"echo 'Compiler launcher detection order check:'",
"which ccache || echo 'ccache not found'",
"which sccache || echo 'sccache not found'",
"echo 'PATH priority test:'",
"for prog in ccache sccache; do echo \"$prog: $(which $prog 2>/dev/null || echo 'not found')\"; done"
]

[tool.cibuildwheel.windows]
before-build = ["choco install ccache"]
environment-pass = ["SCCACHE_GHA_ENABLED", "ACTIONS_RUNTIME_TOKEN", "ACTIONS_RESULTS_URL", "ACTIONS_CACHE_SERVICE_V2", "SCCACHE_C_CUSTOM_CACHE_BUSTER", "PYTHON_GIL"]
# if ccache is in the path, it's chosen before sccache
# and, uninstalling strawberry is much slower than just deleting the entry
before-build = [
"del \"C:\\Strawberry\\c\\bin\\ccache.exe\"",
"choco install sccache",
"sccache --version"
]
before-test = [
"sccache --show-stats"
]

[[tool.scikit-build.overrides]]
# Regular Windows builds (non-free-threading)
if.platform-system = "^win32"
if.abi-flags = ""
cmake.args = [
"-G", "Ninja",
"-DCMAKE_BUILD_TYPE=RelWithDebInfo",
"-DCMAKE_MSVC_DEBUG_INFORMATION_FORMAT=Embedded",
"-DCMAKE_C_FLAGS_RELWITHDEBINFO=/O2 /Z7 /DNDEBUG",
"-DCMAKE_CXX_FLAGS_RELWITHDEBINFO=/O2 /Z7 /DNDEBUG",
"-DCMAKE_SYSTEM_PROCESSOR=AMD64",
"-DCMAKE_C_COMPILER_LAUNCHER=",
"-DCMAKE_CXX_COMPILER_LAUNCHER=",
"-DCORE_EXTENSIONS=",
"-DBUILD_UNITTESTS=OFF"
]

[[tool.scikit-build.overrides]]
# Free-threading Windows builds (Python 3.14t)
# To get sccache to work, needed a few things
# Ninja
# /Z7 so we don't write parallel PDBs
# The empty COMPILER_LAUNCHERS are because otherwise sccache is called twice (sccache sccache cl.exe ...)
# The LIBRARY is because it picks up python314.lib, instead of python314t.lib
# TODO: Dynamically add the "t" for future versions (ie: python315t)
if.platform-system = "^win32"
if.abi-flags = "t"
cmake.args = [
"-G", "Ninja",
"-DCMAKE_BUILD_TYPE=RelWithDebInfo",
"-DCMAKE_MSVC_DEBUG_INFORMATION_FORMAT=Embedded",
"-DCMAKE_C_FLAGS_RELWITHDEBINFO=/O2 /Z7 /DNDEBUG /DPY_NO_LINK_LIB /DPy_GIL_DISABLED",
"-DCMAKE_CXX_FLAGS_RELWITHDEBINFO=/O2 /Z7 /DNDEBUG /DPY_NO_LINK_LIB /DPy_GIL_DISABLED",
"-DCMAKE_C_COMPILER_LAUNCHER=",
"-DCMAKE_CXX_COMPILER_LAUNCHER=",
]
# "-DCMAKE_SYSTEM_PROCESSOR=AMD64",

#"-DCMAKE_SHARED_LINKER_FLAGS=/NODEFAULTLIB:python314.lib",
#"-DCMAKE_EXE_LINKER_FLAGS=/NODEFAULTLIB:python314.lib",

# "-DCMAKE_LIBRARY_PATH=${Python3_LIBRARY_DIRS}",

#"-DPython3_FIND_STRATEGY=LOCATION",
#"-DPython3_FIND_ABI=ANY;ANY;ANY;ON",
#"-DPython3_LIBRARY_RELEASE=python314t.lib",
# "-DCORE_EXTENSIONS=",
# "-DBUILD_UNITTESTS=OFF"
# "--debug-output",
# "--debug-find",
# "--debug-find-pkg=Python3",
# "--trace-expand",
# "--log-level=DEBUG"

11 changes: 11 additions & 0 deletions test_mre.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
import pandas as pd


def test_chained_assignment_warning():
"""This should trigger the ChainedAssignmentError warning we're trying to suppress"""
df = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6]})

# This chained assignment triggers the warning
df['A'][0] = 999

assert df['A'][0] == 999
Loading