Substantial performance regression on 1.11 relative to 1.10

[palday]

Casual testing on my M1 Macbook shows a pretty substantial performance regression on tests in 1.11 relative to 1.10.

Julia 1.10:

Test Summary: |  Pass  Total     Time
Arrow         | 66047  66047  5m08.9s

Julia 1.11:

Test Summary: |  Pass  Total      Time
Arrow         | 66047  66047  13m54.1s

[Moelf]

is this just running ]test? if so it may not be a performance regression on normal usage

[palday]

Why would testing be different?

Also, more than doubling the time it takes to run tests seems ... less than optimal when it comes to development.

[Moelf]

tests are run with stuff like bounds checking forced "on" for everything, it could be some heuristics change from Julia compiler side that accidentally changed compilation result for test-time only

less than optimal when it comes to development.

yes assuming there is development...

[palday]

Manually running tests outside of Pkg.test:

1.10:

julia +1.10 --project=test -e'using Pkg; Pkg.develop(path="./"); Pkg.update(); @time include("test/runtests.jl")'

Test Summary: | Pass  Total  Time
ArrowTypes    |  127    127  0.5s
Test Summary:                       | Pass  Total  Time
Date and time wrappers with missing |    4      4  5.0s
Test Summary:      | Pass  Total  Time
`default(T) isa T` |   10     10  0.0s
Test Summary: |  Pass  Total     Time
Arrow         | 66047  66047  3m48.7s
239.948859 seconds (897.30 M allocations: 59.172 GiB, 5.05% gc time, 97.20% compilation time: <1% of which was recompilation)

1.11:

julia +1.11 --project=test -e'using Pkg; Pkg.develop(path="./"); Pkg.update(); @time include("test/runtests.jl")'

Test Summary: | Pass  Total  Time
ArrowTypes    |  127    127  0.5s
Test Summary:                       | Pass  Total  Time
Date and time wrappers with missing |    4      4  5.9s
Test Summary:      | Pass  Total  Time
`default(T) isa T` |   10     10  0.0s
Test Summary: |  Pass  Total      Time
Arrow         | 66047  66047  11m50.4s
724.425272 seconds (4.33 G allocations: 196.264 GiB, 2.73% gc time, 99.11% compilation time: <1% of which was recompilation

[Moelf]

that's pretty bad huh

[palday]

Possibly related julialang issues:

• Huge precompilation overhead in Julia 1.11 JuliaLang/julia#56080
• Compilation time regression in 1.11 JuliaLang/julia#56093
• Latency regression on first error JuliaLang/julia#56861

[vtjnash]

Seems to be mostly fixed as of v1.12 (looking locally, and at latest CI for example: https://github.com/apache/arrow-julia/actions/runs/17845280009/job/50743369381)

[kou]

Thanks.
Can we close this?