Implement COPY … FROM STDIN queries

[ahoppen]

This implements support for COPY operations using COPY … FROM STDIN queries for fast data transfer from the client to the backend.

Performance

A quick note on local performance measurements: Inserting the numbers from 0 to 1,000,000 into a table that has two columns (INT and VARCHAR) takes ~150ms. Depending on the exact implementation, the majority of the active CPU cycles are spent converting the numbers to strings, inside string interpolation or inside ByteBuffer._setBytes. If I remove the code that sends the CopyData messages to the backend (but keep all other logic that might incur thread hopes), the test described above takes ~50ms and utilizes the CPU at ~200%, so the real bottleneck here is the Postges backend handling the data. For comparison psycopg2 takes 210ms with the data to be written already prepared in a StringIO object. So, performance-wise this PR should be good to go.

Ideas for follow-up PRs

• Check if we should buffer data sent through the PostgresCopyFromWriter to reduce the number of CopyData messages we need to send (and thus the protocol overhead). Alternatively, we can leave that kind of optimization to the client.
• Add an API that allows binary transfer of data
• Implement remaining options that can be passed to COPY FROM.
• Allow concurrently generating the data to be written and flushing a buffer to the backend.

Fixes #290

[fabianfett]

Another review round. Thanks so much for pushing through this!

[fabianfett]

Another round. Please also make sure that merge conflicts are resolved and tests pass

[ahoppen]

Thanks for the review, Fabian 🙂

Rebased, updated tests to use Swift Testing and addressed/replied to the review comments.

[codecov]

Codecov Report

❌ Patch coverage is 83.55438% with 62 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.56%. Comparing base (78114d4) to head (af67821).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
...tion State Machine/ExtendedQueryStateMachine.swift	76.47%	28 Missing ⚠️
...nection State Machine/ConnectionStateMachine.swift	76.56%	15 Missing ⚠️
...urces/PostgresNIO/New/PostgresChannelHandler.swift	85.50%	10 Missing ⚠️
...esNIO/Connection/PostgresConnection+CopyFrom.swift	93.10%	8 Missing ⚠️
Sources/PostgresNIO/New/PSQLTask.swift	80.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #566      +/-   ##
==========================================
+ Coverage   75.22%   75.56%   +0.33%     
==========================================
  Files         132      134       +2     
  Lines        9534     9887     +353     
==========================================
+ Hits         7172     7471     +299     
- Misses       2362     2416      +54     

Files with missing lines	Coverage Δ
...tgresNIO/New/Extensions/AnyErrorContinuation.swift	100.00% <100.00%> (ø)
Sources/PostgresNIO/New/PSQLTask.swift	87.50% <80.00%> (-1.08%)	⬇️
...esNIO/Connection/PostgresConnection+CopyFrom.swift	93.10% <93.10%> (ø)
...urces/PostgresNIO/New/PostgresChannelHandler.swift	90.56% <85.50%> (-0.49%)	⬇️
...nection State Machine/ConnectionStateMachine.swift	68.75% <76.56%> (+0.86%)	⬆️
...tion State Machine/ExtendedQueryStateMachine.swift	78.48% <76.47%> (+0.16%)	⬆️

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[fabianfett]

I think this is good to merge. Anything that comes up now we can do in a follow up.