Skip to content

feat(storage): add variant shredding for parquet#19422

Draft
dantengsky wants to merge 19 commits intodatabendlabs:mainfrom
dantengsky:feat/variant-shredding
Draft

feat(storage): add variant shredding for parquet#19422
dantengsky wants to merge 19 commits intodatabendlabs:mainfrom
dantengsky:feat/variant-shredding

Conversation

@dantengsky
Copy link
Member

@dantengsky dantengsky commented Feb 7, 2026

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

  • Enable Parquet VARIANT encoding + inline shredding for Variant columns in Fuse
  • Add read-path unshred, stats gating, and EXPLAIN inline indication with JSONB compatibility
  • Add unit + sqllogic coverage for typed_value decoding and inline shredding

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions bot added the pr-feature this PR introduces a new feature to the codebase label Feb 7, 2026
@dantengsky dantengsky added the ci-cloud Build docker image for cloud test label Feb 7, 2026
@dantengsky dantengsky force-pushed the feat/variant-shredding branch 5 times, most recently from 62f0cdd to 6b94ea6 Compare February 8, 2026 09:04
@dantengsky dantengsky force-pushed the feat/variant-shredding branch from de9864a to 69f717e Compare February 9, 2026 11:14
@databendlabs databendlabs deleted a comment from github-actions bot Feb 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-cloud Build docker image for cloud test pr-feature this PR introduces a new feature to the codebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant