How can I save partitioned Parquet ? #3737
Answered
by
andygrove
Miyake-Diogo
asked this question in
Q&A
Replies: 2 comments 2 replies
-
Have you tried calling write_parquet on a DataFrame? https://arrow.apache.org/datafusion/user-guide/dataframe.html |
Beta Was this translation helpful? Give feedback.
1 reply
-
let ctx = SessionContext::new();
ctx
.read_csv("data.csv", CsvReadOptions::default().has_header(true)).await?
.repartition(Partitioning::Hash(vec![col("id")], 2))?
.write_parquet("data.parquet", None).await?; |
Beta Was this translation helpful? Give feedback.
1 reply
Answer selected by
Miyake-Diogo
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
How can I save partitioned parquet like apache spark?
Beta Was this translation helpful? Give feedback.
All reactions