GH-47981: [C++][Parquet] Add compatibility with non-compliant RLE stream

[pitrou]

Rationale for this change

RLE-bit-packed streams are required by the Parquet spec to have 8-padded bit-packed runs, but some non-compliant encoders (such as Polars versions before pola-rs/polars#13883) might generate a truncated last bit-packed run, which nevertheless contains enough logical values.

What changes are included in this PR?

1. Compatibility code for non-compliant RLE streams as described above
2. Guard against zero-size dictionaries to avoid hitting an assertion in DictionaryConverter

Are these changes tested?

Yes, by additional unit tests.

Are there any user-facing changes?

No, except a bugfix.

• GitHub Issue: [C++][Python][Parquet] pyarrow.lib.ArrowInvalid: Invalid number of indices: 0 when reading a parquet file #47981

[pitrou]

@AntoinePrv FYI

[pitrou]

@AntoinePrv Are you available to review this?

[pitrou]

@ursabot please benchmark lang=C++

[voltrondatabot]

Benchmark runs are scheduled for commit f613124. Watch https://buildkite.com/apache-arrow and https://conbench.ursa.dev for updates. A comment will be posted here when the runs are complete.

[pitrou]

@ursabot please benchmark lang=C++

[voltrondatabot]

Benchmark runs are scheduled for commit ba8cd20. Watch https://buildkite.com/apache-arrow and https://conbench.ursa.dev for updates. A comment will be posted here when the runs are complete.

[conbench-apache-arrow]

Thanks for your patience. Conbench analyzed the 4 benchmarking runs that have been run so far on PR commit f613124.

There were 9 benchmark results indicating a performance regression:

• Pull Request Run on amd64-c6a-4xlarge-linux at 2025-11-10 11:21:29Z
  • GrouperWithMultiTypes (C++) with params="{fixed_size_binary(32)}"/4096/1, source=cpp-micro, suite=arrow-compute-grouper-benchmark
  • GrouperWithMultiTypes (C++) with params="{fixed_size_binary(32)}"/1024/1, source=cpp-micro, suite=arrow-compute-grouper-benchmark
• and 7 more (see the report linked below)

The full Conbench report has more details.

[conbench-apache-arrow]

Thanks for your patience. Conbench analyzed the 4 benchmarking runs that have been run so far on PR commit ba8cd20.

There were 6 benchmark results indicating a performance regression:

• Pull Request Run on amd64-c6a-4xlarge-linux at 2025-11-10 14:44:52Z
  • GrouperWithMultiTypes (C++) with params="{fixed_size_binary(32)}"/4096/1, source=cpp-micro, suite=arrow-compute-grouper-benchmark
  • GrouperWithMultiTypes (C++) with params="{fixed_size_binary(32)}"/1024/1, source=cpp-micro, suite=arrow-compute-grouper-benchmark
• and 4 more (see the report linked below)

The full Conbench report has more details.

[pitrou]

@ursabot please benchmark lang=C++

[voltrondatabot]

Benchmark runs are scheduled for commit 959327f. Watch https://buildkite.com/apache-arrow and https://conbench.ursa.dev for updates. A comment will be posted here when the runs are complete.

[pitrou]

@github-actions crossbow submit -g cpp

[github-actions]

Revision: 959327f

Submitted crossbow builds: ursacomputing/crossbow @ actions-f55022a5e6

Task	Status
example-cpp-minimal-build-static
example-cpp-minimal-build-static-system-dependency
example-cpp-tutorial
test-build-cpp-fuzz
test-conda-cpp
test-conda-cpp-valgrind
test-cuda-cpp-ubuntu-22.04-cuda-11.7.1
test-debian-12-cpp-amd64
test-debian-12-cpp-i386
test-fedora-42-cpp
test-ubuntu-22.04-cpp
test-ubuntu-22.04-cpp-20
test-ubuntu-22.04-cpp-bundled
test-ubuntu-22.04-cpp-emscripten
test-ubuntu-22.04-cpp-no-threading
test-ubuntu-24.04-cpp
test-ubuntu-24.04-cpp-bundled-offline
test-ubuntu-24.04-cpp-gcc-13-bundled
test-ubuntu-24.04-cpp-gcc-14
test-ubuntu-24.04-cpp-minimal-with-formats
test-ubuntu-24.04-cpp-thread-sanitizer

[conbench-apache-arrow]

Thanks for your patience. Conbench analyzed the 4 benchmarking runs that have been run so far on PR commit 959327f.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details.

[wgtmac]

LGTM. I just have a minor question.

[wgtmac]

Why do we need this change?

[pitrou]

Because kMaxCount uses CeilDiv and is therefore one large than the actual max.

That said, this is a bit confusing so I could try to change to kMaxCount = internal::max_size_for_v<rle_size_t> / 8.

[pitrou]

@github-actions crossbow submit -g cpp

[github-actions]

Revision: 5fe9995

Submitted crossbow builds: ursacomputing/crossbow @ actions-926cb57385

Task	Status
example-cpp-minimal-build-static
example-cpp-minimal-build-static-system-dependency
example-cpp-tutorial
test-build-cpp-fuzz
test-conda-cpp
test-conda-cpp-valgrind
test-cuda-cpp-ubuntu-22.04-cuda-11.7.1
test-debian-12-cpp-amd64
test-debian-12-cpp-i386
test-fedora-42-cpp
test-ubuntu-22.04-cpp
test-ubuntu-22.04-cpp-20
test-ubuntu-22.04-cpp-bundled
test-ubuntu-22.04-cpp-emscripten
test-ubuntu-22.04-cpp-no-threading
test-ubuntu-24.04-cpp
test-ubuntu-24.04-cpp-bundled-offline
test-ubuntu-24.04-cpp-gcc-13-bundled
test-ubuntu-24.04-cpp-gcc-14
test-ubuntu-24.04-cpp-minimal-with-formats
test-ubuntu-24.04-cpp-thread-sanitizer

[conbench-apache-arrow]

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 0c20874.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 7 possible false positives for unstable benchmarks that are known to sometimes produce them.