fix(pandas): respect Optional pydantic fields as nullable in PydanticModel columns (closes #2406)

[feiiiiii5]

Root Cause

When Config.dtype = PydanticModel(Model) is used in a DataFrameModel, _build_pydantic_column() in pandera/api/pandas/container.py created Column objects without checking whether the corresponding pydantic field is Optional. All columns defaulted to nullable=False, causing:

SchemaError: non-nullable series 'rating' contains null values

for fields like rating: Optional[PositiveInt] = None.

This is a regression: worked correctly in 0.32.0 and 0.31.1.

Fix

_build_pydantic_column now inspects the pydantic field's annotation and default value:

• Union with NoneType (e.g. Optional[int], int | None) → nullable=True
• Default value None → nullable=True
• Otherwise → nullable=False (unchanged)

Supports both Pydantic v1 (__fields__) and v2 (model_fields).

Test

• 5 regression tests (tests/pandas/test_pydantic_optional_nullable.py)
  • Optional field is nullable in schema
  • Required field is not nullable in schema
  • Required str field is not nullable
  • Missing optional column accepted
  • Optional column with None values accepted
• All 18 existing pydantic tests still pass

Diff scope

2 files changed, +118/-3 lines (35 lines fix + 83 lines tests)

AI Disclosure

AI-assisted debug/initial draft/testing. Root cause identified from issue #2406 analysis. Tests verified locally. Human review of diff scope and correctness.