Add support for running pandas queries with cudf.pandas enabled#148
Add support for running pandas queries with cudf.pandas enabled#148ritchie46 merged 4 commits intopola-rs:mainfrom
Conversation
|
This needs a rebase. |
d8baf65 to
306e6fb
Compare
|
Done. However, the last release of cudf has an upper bound on the supported version that bumps us back to 1.25 here. I don't know if that is compatible with the polars cloud bits that you recently added. If you prefer, I can revert the changes adding cudf to the environment and we can continue relying on the Note that when we first added GPU benchmarks to this repo cudf was not yet available on PyPI, only NVIDIA's pip index, so there was an even stronger reason not to add it to the environment here. Now that we can get cudf from PyPI it is feasible to do this, with the main issue being if the upper bounds that we impose for stability reasons are prohibitive for your use cases in this repo. Ideally we'd be able to relax that bound eventually, but I don't think we're quite comfortable enough to do that yet. |
|
The 25.06 release of cudf will support Polars 1.28, so perhaps the best option here is to wait for that release so that we don't have to change the supported Polars version here. |
306e6fb to
7fd4cca
Compare
|
@ritchie46 The open question on this PR is that since cudf-polars currently place an upper bound on polars, if cudf-polars is part of the environment then it upper bounds the version of polars we can have installed until the next release. If that is OK, I can update this PR with latest main. Otherwise I can simplify this PR by removing the requirements changes and the |
I think I'd prefer that. Can you also rebase? I think that should satisfy mypi. |
7fd4cca to
0a6c208
Compare
Both done. Unfortunately still seeing mypy errors. I opened #173 to resolve the outstanding issues with CI. |
This PR makes it possible to run the pandas queries with GPU acceleration analogous to the support for the Polars GPU engine. To support this, cudf is added to the requirements list (which means we should also be able to run the Polars GPU engine benchmarks with the virtual environment now).