ARROW-15483: [Release] Revamp the verification scripts #12320
Conversation
|
Thanks for opening a pull request! If this is not a minor PR. Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW Opening JIRAs ahead of time contributes to the Openness of the Apache Arrow project. Then could you also rename pull request title in the following format? or See also: |
|
@github-actions crossbow submit verify-rc-source-cpp-* |
|
Revision: 7e6563f Submitted crossbow builds: ursacomputing/crossbow @ actions-1567
|
|
@github-actions crossbow submit verify-rc-source-cpp-macos-arm64 |
|
Revision: 86cc572 Submitted crossbow builds: ursacomputing/crossbow @ actions-1568
|
|
It would be nice to stop using |
| cmake \ | ||
| curl \ | ||
| git \ | ||
| libcurl4-openssl-dev \ |
|
Side note: perhaps https://docs.pyinvoke.org/en/stable/ could be useful to replace shell / command.com for the release scripts :-) |
|
Maybe in the next iteration (I prefer plumbum by the way). |
|
@github-actions crossbow submit verify-rc-source-*-macos-arm64 |
|
Revision: 03c5a09 Submitted crossbow builds: ursacomputing/crossbow @ actions-1569
|
|
@github-actions crossbow submit verify-rc-source-cpp-macos-arm64 |
|
Revision: 4b894f4 Submitted crossbow builds: ursacomputing/crossbow @ actions-1570
|
|
@github-actions crossbow submit verify-rc-source-cpp-linux-* |
|
Revision: 4b894f4 Submitted crossbow builds: ursacomputing/crossbow @ actions-1571
|
|
@github-actions crossbow submit verify-rc-source-*-linux-almalinux-8-amd64 |
|
Revision: 4b894f4 Submitted crossbow builds: ursacomputing/crossbow @ actions-1574
|
|
@github-actions crossbow submit verify-rc-source-*-linux-almalinux-8-amd64 |
|
Revision: 3148f5e Submitted crossbow builds: ursacomputing/crossbow @ actions-1575
|
|
@github-actions crossbow submit verify-rc-source-*-linux-almalinux-8-amd64 |
|
Revision: b70ac1d Submitted crossbow builds: ursacomputing/crossbow @ actions-1576
|
|
@github-actions crossbow submit verify-rc-source-cpp-linux-conda-amd64 |
|
Revision: b70ac1d Submitted crossbow builds: ursacomputing/crossbow @ actions-1577
|
|
@github-actions crossbow submit verify-rc-source-* |
|
Revision: 89a0790cde0aa1e604c59e87a8ab8504e02dfdad Submitted crossbow builds: ursacomputing/crossbow @ actions-1578
|
kszucs
changed the title
kszucs
force-pushed
the
revamp-verification
branch
from
b4e1c3f to
e202be7
Compare
|
@github-actions crossbow submit verify-rc-source-* |
|
Revision: e202be7 Submitted crossbow builds: ursacomputing/crossbow @ actions-1710
|
|
It's just a test problem not implementation. We can accept Ruby 2.5 with the following patch: diff --git a/ruby/red-arrow/test/test-table.rb b/ruby/red-arrow/test/test-table.rb
index 5d039c9cfa..2cdccb95fd 100644
--- a/ruby/red-arrow/test/test-table.rb
+++ b/ruby/red-arrow/test/test-table.rb
@@ -186,7 +186,12 @@ class TableTest < Test::Unit::TestCase
end
test("{key: Range}: beginless include end") do
- assert_equal(<<-TABLE, @table.slice(count: ..8).to_s)
+ begin
+ range = eval("..8")
+ rescue SyntaxError
+ omit("beginless range isn't supported")
+ end
+ assert_equal(<<-TABLE, @table.slice(count: range).to_s)
count visible
0 1 true
1 2 false
@@ -196,7 +201,12 @@ class TableTest < Test::Unit::TestCase
end
test("{key: Range}: beginless exclude end") do
- assert_equal(<<-TABLE, @table.slice(count: ...8).to_s)
+ begin
+ range = eval("...8")
+ rescue SyntaxError
+ omit("beginless range isn't supported")
+ end
+ assert_equal(<<-TABLE, @table.slice(count: range).to_s)
count visible
0 1 true
1 2 false
@@ -205,7 +215,12 @@ class TableTest < Test::Unit::TestCase
end
test("{key: Range}: endless") do
- assert_equal(<<-TABLE, @table.slice(count: 16..).to_s)
+ begin
+ range = eval("16..")
+ rescue SyntaxError
+ omit("endless range isn't supported")
+ end
+ assert_equal(<<-TABLE, @table.slice(count: range).to_s)
count visible
0 16 true
1 32 false |
kszucs
force-pushed
the
revamp-verification
branch
from
e1f3a06 to
66f824b
Compare
|
@github-actions crossbow submit verify-rc-source-ruby-linux-ubuntu-18.04-amd64 |
|
Revision: 66f824b Submitted crossbow builds: ursacomputing/crossbow @ actions-1714
|
|
@kou should be ready now |
|
@github-actions crossbow submit verify-rc-source-ruby-* |
|
Revision: cfe1f82 Submitted crossbow builds: ursacomputing/crossbow @ actions-1717
|
| set -o pipefail | ||
|
|
||
| if [ ${VERBOSE:-0} -gt 0 ]; then | ||
| set -x |
| # Install dependencies | ||
| if [ $# -gt 0 ]; then | ||
| show_info "Installed pip packages $@..." | ||
| pip install $@ |
There was a problem hiding this comment.
Generally, we should surround $@ with " to handle spaces correctly. (In this case, without " will work.)
| pip install $@ | |
| pip install "$@" |
ruby -e 'p ARGV' $@
ruby -e 'p ARGV' "$@"$ bash /tmp/a.sh 1 "2 3" 4
["1", "2", "3", "4"]
["1", "2 3", "4"]|
|
||
| test_glib() { | ||
| pushd c_glib | ||
| show_header "Build and test C Glib libraries" |
There was a problem hiding this comment.
| show_header "Build and test C Glib libraries" | |
| show_header "Build and test C GLib libraries" |
| setup_conda glib gobject-introspection meson ninja ruby || exit 1 | ||
| setup_virtualenv meson || exit 1 |
There was a problem hiding this comment.
I want to verify C GLib with system libraries (installed by apt).
Could you add support for this case?
We can confirm these dependencies by the following command lines:
$ pkg-config --exists glib-2.0
$ pkg-config --exists gobject-introspection-1.0
$ meson --version
$ ninja --version
$ ruby --versionThere was a problem hiding this comment.
The two cases are mutually exclusive, setup conda will run only if USE_CONDA=1 is set, otherwise the script will use the system dependencies. In the non-conda case we're already a python virtualenv (required for the libarrow_python) we can install meson using pip.
There was a problem hiding this comment.
I renamed:
setup_conda=>maybe_setup_condasetup_virtualenv=>maybe_setup_virtualenvsetup_go=>maybe_setup_gosetup_nodejs=>maybe_setup_nodejs
| setup_conda || exit 1 | ||
| setup_virtualenv || exit 1 |
There was a problem hiding this comment.
Please don't call them if we use system libraries.
BTW, I think that we can omit || exit 1 because we use set -e.
There was a problem hiding this comment.
|| exit 1 is there to handle the return codes from the functions like return 1 which is used to early terminate without actually stopping the script.
| export GOPATH=`pwd`/local-go/gopath | ||
| export PATH=$GOROOT/bin:$GOPATH/bin:$PATH | ||
| setup_nodejs || exit 1 | ||
| setup_conda nodejs=17 || exit 1 |
There was a problem hiding this comment.
I think that this is needless because setup_nodejs will install Node.js.
There was a problem hiding this comment.
I'm updating the name of the setup functions because they are mutually exclusive. Here it means that either install nodejs by downloading it or if we are in a conda environment then install the conda package instead.
| show_header "Build and test Go libraries" | ||
|
|
||
| setup_go || exit 1 | ||
| setup_conda compilers go=1.17 || exit 1 |
There was a problem hiding this comment.
We either setup go by downloading and installing it or install the go package from conda.
One can set GO_ALREADY_INSTALLED=1 to omit installing the non-conda go.
|
@github-actions crossbow submit verify-rc-source-* |
|
Revision: a2acdbb Submitted crossbow builds: ursacomputing/crossbow @ actions-1720
|
|
+1 Can we merge this? |
If you think that the changes are okay than yes :) |
|
Merged: :-) |
|
Benchmark runs are scheduled for baseline = 4ef95eb and contender = a13870e. a13870e is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
4 participants
No description provided.