PYTHON-4700 - Convert CSFLE tests to async

[NoahStapp]

No description provided.

[blink1073]

Why only once?

[NoahStapp]

This is to support KMS requests, which require that the read returns the first batch immediately to accurately update the expected length.

[ShaneHarvey]

That doesn't sound right. If we know how much we need to read we should be looping until we read that amount.

[NoahStapp]

This loop inside _async_kms_request changes kms_context.bytes_needed with each recv and feed call. If we allow async_receive_data_socket to run until the initial kms_context.bytes_needed number of bytes is read, it will result in an error.

while kms_context.bytes_needed > 0:
    # CSOT: update timeout.
    conn.settimeout(max(_csot.clamp_remaining(_KMS_CONNECT_TIMEOUT), 0))
    data = await async_receive_data_socket(conn, kms_context.bytes_needed)
    kms_context.feed(data)

[NoahStapp]

This is the behavior I observed that led to this change:

kms_context.bytes_needed = N
async_receive_data_socket returns with X bytes read, X < N
kms_context.feed is called
kms_context.bytes_needed is now not N - X, but a different value

[ShaneHarvey]

Oh I see, looks like that's intentional behavior in libmongocrypt:

/* Before we've read the Content-Length header in an HTTP response,
 * we don't know how many bytes we'll need. So return this value
 * in kms_ctx_bytes_needed until we are fed the Content-Length.
 */
#define DEFAULT_MAX_KMS_BYTE_REQUEST 1024

https://github.com/mongodb/libmongocrypt/blob/7aeaec4/src/mongocrypt-kms-ctx.c#L47-L51

[NoahStapp]

Yeah, it's a little odd but this approach allows us to reuse as much code as possible.

[blink1073]

Can you add Shane's note as a comment for future readers?

[NoahStapp]

test.asynchronous.test_encryption.AsyncTestSpec.test_legacy_timeoutMS_timeoutMS_applied_to_listCollections_to_get_collection_schema still flakey on Linux even with the increased timeouts: https://spruce.mongodb.com/task/mongo_python_driver_tests_python_version_rhel8_test_encryption__platform~rhel8_auth_ssl~noauth_nossl_python_version~3.10_encryption~encryption_test_6.0_replica_set_patch_ac198af557410bce4809138c0089e1e56ff6db87_6707e960e0a65f0007387fc6_24_10_10_14_49_07?execution=1&sortBy=STATUS&sortDir=ASC

[blink1073]

Maybe we need to skip that test and mark it for PYTHON-4493?

[ShaneHarvey]

Agree. I suggest opening a new ticket for the skipped test to make sure it doesn't get lost.

[NoahStapp]

test.asynchronous.test_encryption.AsyncTestSpec.test_legacy_timeoutMS_timeoutMS_applied_to_listCollections_to_get_collection_schema

Skipped in #1914.

[blink1073]

Somehow that test is still failing.

[NoahStapp]

Somehow that test is still failing.

I messed up the merge, fixed now.

[blink1073]

LGTM!

[blink1073]

LGTM!