Skip to content

Commit d87bc99

Browse files
authored
GH-45582: [Python] Preserve decimal32/64/256 metadata in Schema.metadata (#45583)
### Rationale for this change Before, these types would be interpreted as `"object"` type and therefore the `precision` and `scale` attributes of these types would not be preserved in the `"metadata"` ### What changes are included in this PR? Uses `pa.types.is_decimal` is instead of `isinstance`ing just the `Decimal128Type` to determine a `"decimal"` pandas type ### Are these changes tested? Yes ### Are there any user-facing changes? Yes, but should not break compatibility. * GitHub Issue: #45582 Authored-by: Matthew Roeschke <[email protected]> Signed-off-by: Antoine Pitrou <[email protected]>
1 parent 6068a0b commit d87bc99

File tree

2 files changed

+15
-1
lines changed

2 files changed

+15
-1
lines changed

python/pyarrow/pandas_compat.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,7 @@ def get_logical_type(arrow_type):
8484
return 'list[{}]'.format(get_logical_type(arrow_type.value_type))
8585
elif isinstance(arrow_type, pa.lib.TimestampType):
8686
return 'datetimetz' if arrow_type.tz is not None else 'datetime'
87-
elif isinstance(arrow_type, pa.lib.Decimal128Type):
87+
elif pa.types.is_decimal(arrow_type):
8888
return 'decimal'
8989
return 'object'
9090

python/pyarrow/tests/test_pandas.py

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -598,6 +598,20 @@ def test_decimal_metadata(self):
598598
assert data_column['numpy_type'] == 'object'
599599
assert data_column['metadata'] == {'precision': 26, 'scale': 11}
600600

601+
@pytest.mark.parametrize('typ', [
602+
pa.decimal32,
603+
pa.decimal64,
604+
pa.decimal128,
605+
pa.decimal256,
606+
])
607+
def test_decimal_other_bitwidts(self, typ):
608+
df = pd.DataFrame({'a': [decimal.Decimal('3.14')]})
609+
schema = pa.schema([pa.field('a', type=typ(4, 2))])
610+
table = pa.Table.from_pandas(df, schema=schema)
611+
col_meta = table.schema.pandas_metadata['columns'][0]
612+
assert col_meta['pandas_type'] == 'decimal'
613+
assert col_meta['metadata'] == {'precision': 4, 'scale': 2}
614+
601615
def test_table_column_subset_metadata(self):
602616
# ARROW-1883
603617
# non-default index

0 commit comments

Comments
 (0)