Add deserialization of Bytes -> Decimal#1534
Add deserialization of Bytes -> Decimal#1534jaychia wants to merge 2 commits intojorgecarleitao:mainfrom
Conversation
I don't understand. Why would decimal be encoded in variable length binary? |
7c57f50 to
8e6d836
Compare
Codecov ReportPatch coverage has no change and project coverage change:
Additional details and impacted files@@ Coverage Diff @@
## main #1534 +/- ##
==========================================
- Coverage 83.02% 82.98% -0.05%
==========================================
Files 391 391
Lines 42786 42814 +28
==========================================
+ Hits 35523 35529 +6
- Misses 7263 7285 +22
☔ View full report in Codecov by Sentry. |
|
Hi @ritchie46, apologies for the late reply! Going by the Parquet spec, decimals are actually able to be encoded as See: https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#decimal
|
|
Also need to impl for nested https://github.com/jorgecarleitao/arrow2/blob/main/src/io/parquet/read/deserialize/nested.rs |
Arrow2 already has support for Parquet FixedLenByteArray -> Decimal conversion
This PR adds support for Parquet (variable-length) ByteArray -> Decimal conversion, re-using most of the logic from FixedLenByteArray conversion