Skip to content

allow for process_record() while reading in avro #26

@gityow

Description

@gityow

Feature request:
Could you allow for process_record function while reading in avro? Here is a suggestion.

def __file_to_dataframe(f, schema, process_record=None, **kwargs):

    reader = fastavro.reader(f, reader_schema=schema)
    records = list()
   if preprocess_record:
            records = [process_record(r) for r in avro_reader]
   else:
            records = list(avro_reader)

    return pd.DataFrame.from_records(records, **kwargs)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions