Skip to content

Optimise input and output chunk sizes #11

@the-mysh

Description

@the-mysh

When writing a file, data are loaded in chunks and the output bytes are saved in chunks.

The sizes of input and output chunks cna be specified by the user.
Size of an input chunk is currently expressed in the number of rows from a dataset.
Size of an output chunk is expressed in the number of bytes.

Both should be optimized for different shapes of datasets - combinations of different lengths (numbers of rows) and widths (number of columns, number of images, number of columns in images).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions