Skip to content

docs: add configurable Parquet writer properties for Iceberg sink and compaction#1029

Draft
Copilot wants to merge 2 commits intomainfrom
copilot/update-documentation-iceberg-sink
Draft

docs: add configurable Parquet writer properties for Iceberg sink and compaction#1029
Copilot wants to merge 2 commits intomainfrom
copilot/update-documentation-iceberg-sink

Conversation

Copy link
Contributor

Copilot AI commented Feb 25, 2026

Documents three new compaction.* sink parameters that control Parquet output for both Iceberg sink writes and compaction, and the deprecation of the old node-level storage config.

Description

  • iceberg/maintenance.mdx

    • Added compaction.write_parquet_compression — Parquet codec (uncompressed, snappy, gzip, lzo, brotli, lz4, zstd; default: snappy)
    • Added compaction.write_parquet_max_row_group_rows — max rows per row group (default: 122880)
    • Updated compaction.target_file_size_mb description: now applies to both sink writes and compaction (not just compaction), with explicit default of 1024 MB
    • Both new params note ALTER SINK (dynamic update) support
    • Added deprecation warning: iceberg_compaction_write_parquet_max_row_group_rows storage config → use compaction.write_parquet_max_row_group_rows instead
    • Updated version note to call out v2.9.0 additions
  • iceberg/iceberg-feature-support.mdx

    • Added Feb 2026 timeline entry for configurable Parquet writer properties

Example usage:

CREATE SINK my_iceberg_sink FROM my_table
WITH (
    connector = 'iceberg',
    type = 'append-only',
    'compaction.target_file_size_mb' = '256',
    'compaction.write_parquet_compression' = 'zstd',
    'compaction.write_parquet_max_row_group_rows' = '50000',
    -- ... other required fields ...
);

Related code PR

risingwavelabs/risingwave#24612

Related doc issue

https://github.com/risingwavelabs/risingwave-docs/issues/24614

Checklist

  • I have run the documentation build locally to verify the updates are applied correctly.
  • For new pages, I have updated mint.json to include the page in the table of contents.
  • All links and references have been checked and are not broken.
Original prompt

This section details on the original issue you should resolve

<issue_title>Document: feat(connector): add configurable write-parquet-properties for Iceberg sink and compaction</issue_title>
<issue_description>This issue tracks the documentation update needed for the merged PR risingwavelabs/risingwave-docs#24612.

Source PR URL: risingwavelabs/risingwave#24612
Source PR Merged At: 2026-02-25T11:40:17Z

If it is a major improvement that deserves a new page or a new section in the documentation, please check if we should label it as an experiment feature.</issue_description>

Comments on the Issue (you are @copilot in this section)


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

… compaction

Co-authored-by: kwannoel <47273164+kwannoel@users.noreply.github.com>
Copilot AI changed the title [WIP] Update documentation for configurable write-parquet-properties docs: add configurable Parquet writer properties for Iceberg sink and compaction Feb 25, 2026
Copilot AI requested a review from kwannoel February 25, 2026 11:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Document: feat(connector): add configurable write-parquet-properties for Iceberg sink and compaction

2 participants