-
|
I have a lot of data that is saved in memory/file in chunks. I want to create a row group, read a chunk of data, save it into parquet via ParquetRowGroupWriter then read another chunk of data and append it to the same row group but it seams not supported and in order to append data I need to open ParquetWriter and create new row group for each chunk of data which is a very bad idea. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
No, row groups are immutable per parquet specification. Adding row groups to existing file is a cool feature on its own because most other libraries treat entire parquet files as immutable. |
Beta Was this translation helpful? Give feedback.
No, row groups are immutable per parquet specification.
Adding row groups to existing file is a cool feature on its own because most other libraries treat entire parquet files as immutable.