WIP: Dummy PR to check maint-18.1.0 status #44709
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…44359) ### Rationale for this change This is not happen on Debian GNU/Linux stable and unstable. This is happen only on Debian GNU/Linux testing. So this may be a temporary problem. ### What changes are included in this PR? Create a non-existent path manually as a workaround. ### Are these changes tested? Yes. ### Are there any user-facing changes? Yes. * GitHub Issue: #44358 Authored-by: Sutou Kouhei <[email protected]> Signed-off-by: Sutou Kouhei <[email protected]>
### Rationale for this change Other file systems such as GCS file system and S3 file system are enabled. ### What changes are included in this PR? Enabled Azure file system. ### Are these changes tested? Yes. ### Are there any user-facing changes? Yes. * GitHub Issue: #44347 Authored-by: Sutou Kouhei <[email protected]> Signed-off-by: Sutou Kouhei <[email protected]>
…n wheel-manylinux-*-cp313t-* (#44362) ### Rationale for this change If interactive deb configuration is used, CI is blocked by waiting an user input. ### What changes are included in this PR? Disable interactive deb configuration. ### Are these changes tested? Yes. ### Are there any user-facing changes? No. * GitHub Issue: #44355 Authored-by: Sutou Kouhei <[email protected]> Signed-off-by: Sutou Kouhei <[email protected]>
…ckaging (#44148) ## WIP Creating the PR to validate if / what are the CI failures ### Rationale for this change Once the issue has been merged: - #25118 We do not require numpy as a pyarrow dependency. ### What changes are included in this PR? Remove numpy as a required dependency ### Are these changes tested? Via CI ### Are there any user-facing changes? Yes, pyarrow won't install numpy as a required dependency. * GitHub Issue: #43846 Authored-by: Raúl Cumplido <[email protected]> Signed-off-by: Raúl Cumplido <[email protected]>
…test wheels on Python 3.13 (#44193) ### Rationale for this change Now that numpy, cython and pandas all have releases that support Python 3.13, we can remove the extra index and ensure that we are building our wheels with the released version instead of nightly dev version. * GitHub Issue: #43519 Authored-by: Joris Van den Bossche <[email protected]> Signed-off-by: Raúl Cumplido <[email protected]>
…test_dataset.py (#44370) ### Rationale for this change If pyarrow has been built without dataset test collection errors due to pytest.mark.parametrize requiring dataset. ### What changes are included in this PR? Move dataset requirement outside of test collection by removing it from the parametrization. ### Are these changes tested? Via archery ### Are there any user-facing changes? No * GitHub Issue: #44369 Authored-by: Raúl Cumplido <[email protected]> Signed-off-by: Sutou Kouhei <[email protected]>
…ve macOS 12 pyarrow wheels (#44315) ### Rationale for this change We did just bump MACOSX_DEPLOYMENT_TARGET from 10 to 12 for pyarrow wheels on 18.0.0, we probably should wait a little before dropping. ### What changes are included in this PR? Revert moving macOS deployment target to 12 ### Are these changes tested? Will trigger wheel jobs on archery ### Are there any user-facing changes? Yes, wheels will be available for macOS 12. * GitHub Issue: #44314 Authored-by: Raúl Cumplido <[email protected]> Signed-off-by: Sutou Kouhei <[email protected]>
…4374) ### Rationale for this change Java-jars are currently failing on macOS to install Python 3.13 because `python@ 3.13` (`python` is an alias of `python@ 3.13` now) isn't installed yet. Our link failure workaround is updating already installed `python@*` with `--overwrite` explicitly. `python@ 3.13` (`python`) isn't installed on GitHub Actions runner yet. So `brew bundle` installs `python` without `--overwrite`. ### What changes are included in this PR? Install `python` with `--overwrite` explicitly before `brew bundle`. ### Are these changes tested? Via CI ### Are there any user-facing changes? No * GitHub Issue: #44373 Lead-authored-by: Sutou Kouhei <[email protected]> Co-authored-by: Raúl Cumplido <[email protected]> Signed-off-by: Sutou Kouhei <[email protected]>
### Rationale for this change When adding Python 3.13 support (#43539), conda was not yet ready to support Python 3.13. But over the last weeks most conda packages having being updated, so trying again. * GitHub Issue: #43519 Authored-by: Joris Van den Bossche <[email protected]> Signed-off-by: Raúl Cumplido <[email protected]>
…cation jobs (#44382) ### Rationale for this change Verification job is currently failing for Ruby. ### What changes are included in this PR? We can't use Python 3.13 and `glib==2.80.5`. So we use old Python as a workaround. ### Are these changes tested? Via archery ### Are there any user-facing changes? No * GitHub Issue: #44381 Lead-authored-by: Sutou Kouhei <[email protected]> Co-authored-by: Raúl Cumplido <[email protected]> Signed-off-by: Raúl Cumplido <[email protected]>
…fication when using Conda (#44388) ### Rationale for this change Conda Python 3.13 fails when running verification tasks for Java. It's related to JPype. See also: * GH-44389 * https://discuss.python.org/t/api-for-python-3-13-prevents-use-of-3rd-party-gc-allocators/62709/5 ### What changes are included in this PR? Pin Python 3.12 on Conda integration jobs because JPype doesn't work with Python 3.13. We can use Python 3.13 again once JPype supports Python 3.13. ### Are these changes tested? Yes via CI ### Are there any user-facing changes? No * GitHub Issue: #44386 Lead-authored-by: Raúl Cumplido <[email protected]> Co-authored-by: Sutou Kouhei <[email protected]> Signed-off-by: Sutou Kouhei <[email protected]>
### Rationale for this change jemalloc may have a problem on ARM. See also: #44342 ### What changes are included in this PR? * Disable jemalloc by default on ARM. * Disable jemalloc for manylinux wheel for ARM. ### Are these changes tested? Yes. ### Are there any user-facing changes? Yes. * GitHub Issue: #44342 Authored-by: Sutou Kouhei <[email protected]> Signed-off-by: Raúl Cumplido <[email protected]>
…rification script (#44407) ### Rationale for this change We can remove these pins now that conda-forge/glib-feedstock#191 has been closed. I've tested on my own crossbow and will test here momentarily. This effectively reverts #44270 and #44268. ### What changes are included in this PR? Tweaks to release release verification script, specifically removing recently-added pins. ### Are these changes tested? Yes: https://github.com/amoeba/crossbow/actions/runs/11335070956/job/31522511285. ### Are there any user-facing changes? No. Closes #44415. * GitHub Issue: #44415 Authored-by: Bryce Mecum <[email protected]> Signed-off-by: Bryce Mecum <[email protected]>
…44425) ### Rationale for this change It seems that the current `quay.io/centos/centos:stream9` for `linux/aarch64` is broken. We use it for building and testing but testing is only broken for now. Because we use cached (a bit old) `quay.io/centos/centos:stream9` for now. Building will be failed when the cache is expired. ### What changes are included in this PR? Change upload phase to after the build from after the test. Test is still failing but we can proceed our release process by this. ### Are these changes tested? Yes. ### Are there any user-facing changes? No. * GitHub Issue: #44422 Authored-by: Sutou Kouhei <[email protected]> Signed-off-by: Raúl Cumplido <[email protected]>
### Rationale for this change See #44353 `UnionMapWriter` does not override `map()` and falls back to implementation in `UnionListWriter` which causes exception when used. ### What changes are included in this PR? Implement `map()` for `UnionMapWriter`. ### Are these changes tested? Yes. ### Are there any user-facing changes? Changed behavior of the related function. * GitHub Issue: #44353 Authored-by: 0xderek <[email protected]> Signed-off-by: David Li <[email protected]>
…44377) ### Rationale for this change See #44361. This allows testing compatibility of the .NET Flight implementation with other Flight implementations. ### What changes are included in this PR? * Adds a new `Apache.Arrow.Flight.IntegrationTest` project that can run in server or client mode for Flight integration tests. * Includes the integration tests that send then retrieve data defined in JSON files, but doesn't add any of the named scenarios * Configures archery to include C# in the Flight integration tests, but skip all the named scenarios * Also skips tests that use dictionary data due to #38045, and the empty data test due to #44363 ### Are these changes tested? These changes are tests. ### Are there any user-facing changes? No * GitHub Issue: #44361 Authored-by: Adam Reeve <[email protected]> Signed-off-by: Curt Hagenlocher <[email protected]>
…ntation (#44424) ### Rationale for this change See #44360 ### What changes are included in this PR? * Adds a new integration test to allow testing `do_exchange` between C++/Python and .NET. * Updates the Flight stream reader to handle when a descriptor is sent in the first message without any schema. ### Are these changes tested? * Yes, using the new integration test. ### Are there any user-facing changes? No * GitHub Issue: #44360 Authored-by: Adam Reeve <[email protected]> Signed-off-by: Curt Hagenlocher <[email protected]>
…source-integration-linux-conda-latest-amd64 (#44395) ### Rationale for this change Because JPype doesn't support Python 3.13 yet. See also: * GH-44389 * https://discuss.python.org/t/api-for-python-3-13-prevents-use-of-3rd-party-gc-allocators/62709/5 ### What changes are included in this PR? Pin Python 3.12. ### Are these changes tested? Yes. ### Are there any user-facing changes? No. * GitHub Issue: #44389 Lead-authored-by: Raúl Cumplido <[email protected]> Co-authored-by: Sutou Kouhei <[email protected]> Signed-off-by: Raúl Cumplido <[email protected]>
### Rationale for this change Executable scripts should have shebang. ### What changes are included in this PR? Add `#!/usr/bin/env bash`. ### Are these changes tested? Yes. ### Are there any user-facing changes? No. * GitHub Issue: #44453 Authored-by: Sutou Kouhei <[email protected]> Signed-off-by: Jacob Wujciak-Jens <[email protected]>
### Rationale for this change `dev/release/06-java-upload.sh` sometimes failed with the following HTTP error: ```text [ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.7:deploy-file (default-cli) on project standalone-pom: Failed to deploy artifacts: Could not transfer artifact org.apache.arrow:arrow-compression:pom:18.0.0 from/to apache.releases.https (https://repository.apache.org/service/local/staging/deploy/maven2): transfer failed for https://repository.apache.org/service/local/staging/deploy/maven2/org/apache/arrow/arrow-compression/18.0.0/arrow-compression-18.0.0.pom, status: 408 Request Timeout -> [Help 1] ``` ### What changes are included in this PR? Use `-DretryFailedDeploymentCount=10` for retry on HTTP error. ### Are these changes tested? Yes. ### Are there any user-facing changes? No. * GitHub Issue: #44451 Authored-by: Sutou Kouhei <[email protected]> Signed-off-by: Jacob Wujciak-Jens <[email protected]>
…ing binary wheels (#44460) ### Rationale for this change Currently the folder used to verify the contents of the wheels is not correct when doing a release verification of the wheels and the verification fails. The script is supposed to check wheels individually and we are already verifying the contents on the wheels individually when generating the wheels. ### What changes are included in this PR? Stop checking wheel content on release verification as the wheel content has already been validated when generating the wheels. There is also a minor fix on the platform tags for the macOS wheels that was missed when we updated the MACOSX_DEPLOYMENT_TARGET. ### Are these changes tested? Yes via archery. ### Are there any user-facing changes? No * GitHub Issue: #44459 Authored-by: Raúl Cumplido <[email protected]> Signed-off-by: Sutou Kouhei <[email protected]>
### Rationale for this change Artifactory sometimes reports the following 503 HTTP error: ```text BinaryTask::ArtifactoryClient::Error: failed to request: https://apache.jfrog.io/artifactory/arrow/debian-rc/pool/bookworm/main/a/apache-arrow/libparquet-glib1800_18.0.0-1_arm64.deb: PUT: 503 { "errors" : [ { "status" : 503, "message" : "503 : Failed during addStream to bucket" } ] } ``` It seems that this is a temporary error. So we can retry on this type error for stable upload. ### What changes are included in this PR? Retry on HTTP error. ### Are these changes tested? Yes. ### Are there any user-facing changes? No. * GitHub Issue: #44449 Authored-by: Sutou Kouhei <[email protected]> Signed-off-by: Sutou Kouhei <[email protected]>
### Rationale for this change Adopting #40226. The creation and return of a shared_ptr does result in some performance overhead, that makes a difference for a performance-sensitive application. If someone could use ChunkResolver to learn the indices, they could then instead access the data directly. ### What changes are included in this PR? - [X] Updates to documentation (thanks to @ SChakravorti21 ) - [X] Moving `ChunkResolver` to public API, and updating all references to it in the code ### Are these changes tested? There seemed to be comprehensive tests already: https://github.com/apache/arrow/blob/main/cpp/src/arrow/chunked_array_test.cc#L324 If an edgecase is missing, I'd be happy to add it. ### Are there any user-facing changes? `ChunkResolver` and `TypedChunkLocation` are now in the public API. * GitHub Issue: #34535 Lead-authored-by: Anja Kefala <[email protected]> Co-authored-by: anjakefala <[email protected]> Co-authored-by: Bryce Mecum <[email protected]> Co-authored-by: SChakravorti21 <[email protected]> Co-authored-by: SChakravorti21<[email protected]> Co-authored-by: Jacob Wujciak-Jens <[email protected]> Signed-off-by: Felipe Oliveira Carvalho <[email protected]>
…44505) ### Rationale for this change Since ChunkLocation is a typedef/alias, it needs to use the `doxygentypedef` Sphinx directive rather than `doxygenstruct` in order for it to show up in the generated HTML. Otherwise we get an error. ### What changes are included in this PR? Just docs: - Replace `doxygenstruct` directive with `doxygentypedef` - Remove `:members:` directive as it's not needed ### Are these changes tested? Yes. ### Are there any user-facing changes? Yes, though just docs. * GitHub Issue: #44506 Authored-by: Bryce Mecum <[email protected]> Signed-off-by: Bryce Mecum <[email protected]>
…cross-compiling (#44507) ### Rationale for this change We can't use `find_package(gRPC)` and `gRPC::grpc_cpp_plugin` for cross-compiling because it's for host. We need `grpc_cpp_plugin` for target in cross-compiling. ### What changes are included in this PR? Add `ARROW_GRPC_CPP_PLUGIN` CMake option that overwrites `gRPC::grpc_cpp_plugin` path found by `find_package(gRPC)`. ### Are these changes tested? Yes. conda-forge/arrow-cpp-feedstock#1432 ### Are there any user-facing changes? Yes. * GitHub Issue: #44448 Authored-by: Sutou Kouhei <[email protected]> Signed-off-by: Sutou Kouhei <[email protected]>
…y-release-candidate-wheels.bat (#44462) ### Rationale for this change The Windows wheel verification fails due to missing `PARQUET_TEST_DATA` ### What changes are included in this PR? Add `PARQUET_TEST_DATA` to `verify-release-candidate-wheels.bat` which is only tested on the binary verification job. ### Are these changes tested? Via archery ### Are there any user-facing changes? No * GitHub Issue: #44461 Lead-authored-by: Sutou Kouhei <[email protected]> Co-authored-by: Raúl Cumplido <[email protected]> Co-authored-by: Sutou Kouhei <[email protected]> Signed-off-by: Raúl Cumplido <[email protected]>
…on (#44613) ### Rationale for this change Abseil's recommended usage is to build from trunk, this version does not provide a package version in CMake which breaks `resolve_dependency`. ### What changes are included in this PR? Add `FindabslAlt.cmake` that adds a big version if it is missing to compare against our minimal version requirements. ### Are these changes tested? CI ### Are there any user-facing changes? Abseil build from head should now work without workaround. * GitHub Issue: #44606 Lead-authored-by: Jacob Wujciak-Jens <[email protected]> Co-authored-by: Sutou Kouhei <[email protected]> Signed-off-by: Jacob Wujciak-Jens <[email protected]>
amoeba
requested review from
CurtHagenlocher,
lidavidm and
trxcllnt
as code owners
|
Thanks for opening a pull request! If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project. Then could you also rename the pull request title in the following format? or In the case of PARQUET issues on JIRA the title also supports: See also: |
github-actions
bot
added
Component: R
Component: Java
Component: Parquet
Component: C++
Component: Python
Component: JavaScript
Component: C#
Component: Ruby
Component: FlightRPC
Component: Gandiva
Component: GLib
Component: MATLAB
Component: Documentation
Component: Swift
awaiting review
Member
Author
|
@github-actions crossbow submit --group verify-rc-source |
Member
Author
|
@github-actions crossbow submit --group packaging |
|
Revision: ac81eed Submitted crossbow builds: ursacomputing/crossbow @ actions-090e5eb390
|
|
Revision: ac81eed Submitted crossbow builds: ursacomputing/crossbow @ actions-201b77b018
|
Member
Author
|
@github-actions crossbow submit verify-rc-source-integration-macos-amd64 |
|
Revision: ac81eed Submitted crossbow builds: ursacomputing/crossbow @ actions-52008bf2a1
|
Member
Author
|
Everything looks good except |
Member
Author
|
verify-rc-source-integration-macos-amd64 is failing because JPype doesn't yet work on Python 3.13, see #44395. Since the intention of that job is to test without conda, I think we can just let it fail for now and move forward with an RC. |
Member
Author
|
Closing now that RC2 is up. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Caution
Do not merge this PR.
This PR is being used to test verification and binary packaging jobs prior to creating 18.1.0 RC2. After testing, we'll close this without merging.