Skip to content

Conversation

@abonander
Copy link
Contributor

@abonander abonander commented Dec 12, 2025

Summary

  • Adds Client::insert_formatted_with() and InsertFormatted for inserting data in a chosen format with a specified SQL query.
    • InsertFormatted does not buffer data by default, allowing precise control over when the data is sent. For best performance, ensure data is sent in larger chunks or use .buffered() to get BufInsertFormatted which implements buffering.
    • Data may optionally be pre-compressed and buffered separately using CompressedData::new() and InsertFormatted::send_compressed().
    • BufInsertFormatted also implements tokio::io::AsyncWrite for composability.
      • Omitted AsyncWrite on InsertFormatted itself because every write would require copying to a new allocation anyway to send to the connection.

closes #339

Checklist

Delete items not relevant to your PR:

  • Unit and integration tests covering the common scenarios were added
  • A human-readable description of the changes was provided so that we can include it in CHANGELOG later

Potential additions:

@mshustov mshustov requested review from serprex and slvrtrn December 15, 2025 13:04
/// Any other type of statement may produce incorrect results.
///
/// The statement is not issued until the first call to `.poll_write()`.
pub fn insert_formatted_with(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps then we need to deprecate fetch_bytes, replacing it with fetch_formatted_with for consistency

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could just be insert_bytes_with, I'm not particularly attached to the naming. I just tried to come up with something more descriptive.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also prefer _formatted_with naming, so it is a good pick!
Let's just keep the naming consistent, i.e., insert_formatted_with + fetch_formatted_with both look good in combination, let's just mark fetch_bytes as deprecated for the next minor release to reduce confusion, and make fetch_formatted or fetch_formatted_with with the same exact content. Also perhaps that new method will be a good candidate to introduce (non-exaustive) enums for common formats, see #312

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this context, the _with suffix is referring to the SQL argument, but on the query side, that would have already been passed to Client::query().

I would call the new method fetch_formatted or fetch_with_format.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created #369 to follow up because this PR is blocking work on the ADBC driver and I want to avoid scope creep.

@codecov
Copy link

codecov bot commented Dec 30, 2025

Codecov Report

❌ Patch coverage is 87.26790% with 48 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/insert_formatted.rs 87.61% 27 Missing and 14 partials ⚠️
src/insert.rs 78.26% 1 Missing and 4 partials ⚠️
src/request_body.rs 88.23% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

serprex
serprex previously approved these changes Dec 31, 2025
@abonander abonander force-pushed the ab/insert-formatted branch from be7929e to 380c032 Compare January 5, 2026 16:01
@abonander
Copy link
Contributor Author

abonander commented Jan 5, 2026

Ugh, the CodeCov failure is due to 7 lines that I'm not even entirely sure how to hit: https://app.codecov.io/gh/ClickHouse/clickhouse-rs/pull/364?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=checks&utm_campaign=pr+comments&utm_term=ClickHouse

We'd have to produce records fast enough to saturate the connection.

There's lines I can try to cover in src/insert_formatted.rs to compensate.

@abonander abonander requested a review from serprex January 8, 2026 03:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Insert data as raw bytes in chosen format

4 participants