Skip to content

Buffered reading #18

@waralex

Description

@waralex

Currently, when reading each block, arrays with columns are created again. And this looks like an extra allocation. There is a suggestion to create them in advance and just do resize when reading the block. Given that blocks are usually about the same size + resize to a smaller size does not cause reallocation this should greatly reduce the number of allocations.

As a bonus, the read and write functions will get the same signatures.

read_col_data(sock::ClickHouseSock, data::T, ::Val{N}, args...) where {T, N}
write_col_data(sock::ClickHouseSock, data::T, ::Val{N}, args...) where {T, N}

And we will be able to read one Сlickhouse type to different Julia types using read_col_data overloads by T and N.
This in turn will give the advanced users great flexibility and the ability to create their own types and fill them directly from the clickhouse without intermediate transformations and arrays.

What do you think of this idea?

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions