Get dtypes from Schema

[chris-wright-nl]

Hi,

My final aim is to be able to enforce pandas to use specific data types at the point of ingestion i.e.:

df = pd.read_csv('my_data.csv', dtype = {'col1': str, 'col2': float, 'col3': int})

I would like to get the dtype from the schema model and then add a class method to the schema so that i can the do:

df = Schema.read_csv("foo.csv")

so far i can get the entries in the schema using:

entries = cls.__annotations__.items()

which returns ['col_1', pandera.typing.pandas.Series[pandas.core.arrays.integer.Int64Dtype] but i'm having trouble accessing the pandas type.

Any thoughts?

[mpg-data]

Hello! Assuming you have a DataframeSchema named schema, then use the following dictionary comprehension for the dtype argument:

{col: dtype_pa.type for col, dtype_pa in schema.dtypes.items()}

[ropeladder]

Getting the dtypes from the schema sort of works, but the validation is provides isn't very helpful: pandas doesn't tell you which row or column breaks the import.

Some kind of wrapper or something that worked more directly with read_csv would be nice.