feat: add decimal value representation

[zhjwpku]

No description provided.

[zhjwpku]

This is ready for review. I've extracted the Decimal128 implementation and tests from Arrow. Please help to review. @wgtmac @lidavidm @mapleFU

[wgtmac]

Could you explicitly mark which part is exactly ported from Arrow to make the current review and future sync process easier?

[zhjwpku]

Could you explicitly mark which part is exactly ported from Arrow to make the current review and future sync process easier?

That's not easy, Arrow's Decimal type uses a complex inheritance hierarchy and supports bit lengths in any 64-bit multiple. Since we only need Decimal128, l omitted the inheritance layer. But the algorithms adapted from Arrow is quite straightforward and should be easy for future sync.

[mapleFU]

What about a thirdparty/ vendor directory, and we can use it same as <arrow/xxx> but it's vendored by us? (Current solving is also ok to me)

[wgtmac]

I guess we cannot port arrow/util/decimal.h without significant change due to dependency on arrow utilities.

My concern is that we use std::array<uint64_t, 2> as the internal representation but use int128_t for testing. I'm not sure if we can directly use int128_t as the physical representation for the Decimal class. I did exactly this when I worked for my previous employer and it should work well.

[zhjwpku]

I guess we cannot port arrow/util/decimal.h without significant change due to dependency on arrow utilities.

My concern is that we use std::array<uint64_t, 2> as the internal representation but use int128_t for testing. I'm not sure if we can directly use int128_t as the physical representation for the Decimal class. I did exactly this when I worked for my previous employer and it should work well.

uint128_t is now used for Decimal multiplier, I think we can use int128_t for the division, let me try the int128_t physical representation idea for the next few days.

[wgtmac]

Then please throw IcebergError and add a comment with \throw by recommending Decimal::FromString to be used in production.

[zhjwpku]

@mapleFU Can you take a look at this again, thanks.

[wgtmac]

This is mainly ported from Arrow C++. Could you help take a quick pass? @lidavidm @zeroshade

[Fokko]

This looks good to me @zhjwpku Thanks for working on this, and thanks @wgtmac and @dongxiao1198 for the review 🙌

[Fokko]

Keep in mind that Iceberg might also store the Decimal as an int32 or int64:

[zhjwpku]

Thanks for the reminder, will keep in mind, thanks.