Metadata on index fields is not persisted

[Girmii]

Describe the bug
The metadata dictionary that can be provided to the pandera.Field model component is not persisted for fields annotated as Index, while it is for fields annotated as Series.

This issue occurs for the metadata field property, but not for the title field property.
I haven't checked any other field properties.

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of pandera.
• (optional) I have confirmed this bug exists on the main branch of pandera.

Code Sample, a copy-pastable example

import pandera.pandas as pa
from pandera.typing import Index, Series


class MyModel(pa.DataFrameModel):
    index_field: Index[float] = pa.Field(title="Index Field", metadata={"test_key1": "test_value1"})
    column_field: Series[float] = pa.Field(
        title="Column Field", metadata={"test_key2": "test_value2"}
    )


class_schema = MyModel.to_schema()

column_field_info = class_schema.columns[MyModel.column_field]
print(f"Column field title: '{column_field_info.title}'")
print(f"Column field metadata: '{column_field_info.metadata}'")

index_field_info = class_schema.index
print(f"Index field title: '{index_field_info.title}'")
print(f"Index field metadata: '{index_field_info.metadata}'")

Output

Column field title: 'Column Field'
Column field metadata: '{'test_key2': 'test_value2'}'
Index field title: 'Index Field'
Index field metadata: 'None'

Expected behavior

I would expect the code snippet to have the following output;

Column field title: 'Column Field'
Column field metadata: '{'test_key2': 'test_value2'}'
Index field title: 'Index Field'
Index field metadata: '{'test_key1': 'test_value1'}'

Versions:

• Python 3.12.0
• pandas 3.0.0
• pandera 0.29.0

[kallenijs]

Just to confirm, would the following test assert your desired fix?

import pandera.pandas as pa
from pandera.typing import Index


def test_index_field_metadata_persistence():
    test_metadata = {"test_key": "test_value"}
    class MyModel(pa.DataFrameModel):
        index_field: Index[float] = pa.Field(title="Index Field", metadata=test_metadata)
    
    class_schema = MyModel.to_schema()

    index_field_info = class_schema.index
    assert index_field_info.metadata == test_metadata 

[kallenijs]

Just to confirm, would the following test assert your desired fix?

import pandera.pandas as pa
from pandera.typing import Index


def test_index_field_metadata_persistence():
    test_metadata = {"test_key": "test_value"}
    class MyModel(pa.DataFrameModel):
        index_field: Index[float] = pa.Field(title="Index Field", metadata=test_metadata)
    
    class_schema = MyModel.to_schema()

    index_field_info = class_schema.index
    assert index_field_info.metadata == test_metadata 

This test fails on main, which would confirm:

• (optional) I have confirmed this bug exists on the main branch of pandera.

[Girmii]

import pandera.pandas as pa
from pandera.typing import Index

def test_index_field_metadata_persistence():
test_metadata = {"test_key": "test_value"}
class MyModel(pa.DataFrameModel):
index_field: Index[float] = pa.Field(title="Index Field", metadata=test_metadata)

class_schema = MyModel.to_schema()

index_field_info = class_schema.index
assert index_field_info.metadata == test_metadata

Indeed it would. If you want to go as minimal as possible, you can leave out the title="Index Field" in the field definition.

[kallenijs]

My friend solved the issue and is committing a PR.

However, looking at the code raised some questions about which behavior is intended, as the Index Field misses several keys relative to the Series Field:

• metadata
• regex
• required

Is there a reason for Index Fields missing these attributes?

The PR includes fix and test for the metadata attribute, but need Maintainer to decide which behavior is desired.

[WPDOrdina]

PR has been committed!