Skip to content

Conversation

ShelbyZ
Copy link
Contributor

@ShelbyZ ShelbyZ commented Aug 28, 2025

Summary

As part of adding support for arrow/parquet compression with S3 we need to install required libraries. These libraries were taken from the install steps for Apache Arrow for AL2023 - https://arrow.apache.org/install/.

PR - Adding Parquet - fluent/fluent-bit#10691

Testing

Ran local latest/debug images for AL2023 (custom build)

Validate release build flags

dev-dsk-shelbyzh-2c-4b7a8c60 % docker run -it amazon/aws-for-fluent-bit:latest-al2023 /fluent-bit/bin/fluent-bit -help | grep "Build Flags" | tr ' ' '\n' | grep ARROW
FLB_HAVE_ARROW
FLB_HAVE_ARROW_PARQUET

Validate debug build flags

dev-dsk-shelbyzh-2c-4b7a8c60 % docker run -it amazon/aws-for-fluent-bit:debug-al2023 /fluent-bit/bin/fluent-bit -help | grep "Build Flags" | tr ' ' '\n' | grep ARROW
FLB_HAVE_ARROW
FLB_HAVE_ARROW_PARQUET

make debug succeeded: yes
Integ tests succeeded: yes
New tests cover the changes: no

Description for the changelog

Enhancement: Add support for Apache Arrow/Parquet for AL2023

Licensing

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@ShelbyZ ShelbyZ requested a review from a team as a code owner August 28, 2025 21:02
@ShelbyZ ShelbyZ added the fluent-bit 4.x Part of the fluent-bit 4.x migration work label Aug 28, 2025
@ShelbyZ ShelbyZ added this to the fluent-bit 4.x milestone Aug 28, 2025
@ShelbyZ ShelbyZ merged commit e498099 into aws:mainline Sep 2, 2025
@ShelbyZ ShelbyZ deleted the arrow-parquet branch September 2, 2025 22:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fluent-bit 4.x Part of the fluent-bit 4.x migration work
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants