GH-45427: [Python] Fix version comparison in pandas compat for pandas 2.3 dev version

[jorisvandenbossche]

Rationale for this change

Small follow-up on #45383 to ensure this version comparison also does the right thing for the currently not-yet-released dev version of 2.3.0

• GitHub Issue: [Python] Fix version comparison in pandas compat for pandas 2.3 dev version #45427

[github-actions]

⚠️ GitHub issue #45427 has been automatically assigned in GitHub to PR creator.

[jorisvandenbossche]

@github-actions crossbow submit -g python

[github-actions]

Revision: a6e4d77

Submitted crossbow builds: ursacomputing/crossbow @ actions-99e2368119

Task	Status
example-python-minimal-build-fedora-conda
example-python-minimal-build-ubuntu-venv
test-conda-python-3.10
test-conda-python-3.10-hdfs-2.9.2
test-conda-python-3.10-hdfs-3.2.1
test-conda-python-3.10-pandas-latest-numpy-latest
test-conda-python-3.10-substrait
test-conda-python-3.11
test-conda-python-3.11-dask-latest
test-conda-python-3.11-dask-upstream_devel
test-conda-python-3.11-hypothesis
test-conda-python-3.11-pandas-latest-numpy-1.26
test-conda-python-3.11-pandas-latest-numpy-latest
test-conda-python-3.11-pandas-nightly-numpy-nightly
test-conda-python-3.11-pandas-upstream_devel-numpy-nightly
test-conda-python-3.11-spark-master
test-conda-python-3.12
test-conda-python-3.12-cpython-debug
test-conda-python-3.13
test-conda-python-3.9
test-conda-python-3.9-pandas-1.1.3-numpy-1.19.5
test-conda-python-emscripten
test-cuda-python-ubuntu-22.04-cuda-11.7.1
test-debian-12-python-3-amd64
test-debian-12-python-3-i386
test-fedora-39-python-3
test-ubuntu-22.04-python-3
test-ubuntu-22.04-python-313-freethreading
test-ubuntu-24.04-python-3

[jorisvandenbossche]

The failing builds are also failing on the latest nightly run

[raulcd]

@jorisvandenbossche did you found out when testing on pandas side?
Probably here, right?
pandas-dev/pandas#60856
Just to get the reference on where was this found.
Not super relevant but to understand where was this coming from.

[jorisvandenbossche]

Yes, that PR and in general the pyarrow nightly CI on the 2.3 branch (or so any PR targetting that). We adjusted our tests for the case pyarrow >= 19 is installed, but that started failing because pyarrow now no longer returns the string dtype for 2.3.0.dev

[conbench-apache-arrow]

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 8fed34e.

There were 8 benchmark results with an error:

• Commit Run on amd64-m5-4xlarge-linux at 2025-02-05 16:20:28Z
  • tpch (R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-03, scale_factor=1
  • tpch (R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-03, scale_factor=10
• and 6 more (see the report linked below)

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 2 possible false positives for unstable benchmarks that are known to sometimes produce them.