-
Notifications
You must be signed in to change notification settings - Fork 70
feat: implement basic parquet writer and add roundtrip tests #198
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR implements a basic ParquetWriter class for writing ArrowArray data to Parquet files. The implementation follows the Writer interface pattern and uses Apache Arrow/Parquet libraries for the underlying functionality.
- Adds ParquetWriter class with PIMPL pattern for encapsulation
- Implements core writer operations: Open, Write, Close with Arrow/Parquet integration
- Provides placeholder implementations for metrics, length, and split_offsets methods
Reviewed Changes
Copilot reviewed 3 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| src/iceberg/parquet/parquet_writer.h | Header file defining the ParquetWriter class interface with PIMPL pattern |
| src/iceberg/parquet/parquet_writer.cc | Implementation of ParquetWriter with Arrow/Parquet integration and factory registration |
| src/iceberg/CMakeLists.txt | Adds parquet_writer.cc to the build configuration |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
602b681 to
978dfcf
Compare
3039e80 to
90039fb
Compare
1f9507a to
02a05cc
Compare
wgtmac
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Left some nits.
Add parquet writer factory and basic parquet writer without metrics.