-
Notifications
You must be signed in to change notification settings - Fork 516
Cherrypicks to aio connector part8 #2458
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: dev/aio-connector
Are you sure you want to change the base?
Cherrypicks to aio connector part8 #2458
Conversation
c290ee1
to
9acde0c
Compare
2a0f804
to
e8f0564
Compare
609427f
to
511d01e
Compare
@@ -47,7 +47,6 @@ def test_valid_pat(wiremock_client: WiremockClient) -> None: | |||
) | |||
|
|||
cnx = snowflake.connector.connect( | |||
user="testUser", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this change was missed in the async code
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
@@ -74,7 +73,6 @@ def test_invalid_pat(wiremock_client: WiremockClient) -> None: | |||
|
|||
with pytest.raises(snowflake.connector.errors.DatabaseError) as execinfo: | |||
snowflake.connector.connect( | |||
user="testUser", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
as above
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
c, "_timebomb", new_callable=MagicMock | ||
) as mock_timerbomb: | ||
mock_timerbomb.executed = True | ||
with mock.patch( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We have this test in async code called test_timeout_query as well.
It still contains the below line:
_with mock.patch.object(c, "timebomb", mock_timebomb):
which was removed due to being identified as a source of flakiness.
Should we move this change to async code?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Timebombs are different objects in sync and async, so we should be fine (as long as mock.patch.object
itself was flaky)
@@ -37,8 +36,6 @@ | |||
|
|||
logger = getLogger(__name__) | |||
|
|||
getLogger("aiohttp").addFilter(AzureCredentialFilter()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't we add aiohttp to MODULES_TO_MASK_LOGS_NAMES in externals_setup.py as well as aioboto3 -> we do not control their logs and there might be some HTTP related logs which can accidently include signatures/keys
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed
@@ -42,9 +45,10 @@ | |||
@pytest.mark.parametrize( | |||
"from_path", [True, pytest.param(False, marks=pytest.mark.skipolddriver)] | |||
) | |||
def test_put_get_with_aws(tmpdir, conn_cnx, from_path): | |||
def test_put_get_with_aws(tmpdir, conn_cnx, from_path, caplog): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would convert those to async as well. - since we know we are using different library (especially unlike requests - aiohttp is not using our vendored urllib3 under the hood - which would be masked using the already present MODULES_TO_MASK_LOGS_NAMES).
We should ensure in tests that async code do not leak sensitive data as well. This comment can be applied to most tests changes from the commit b2ae3e2 .
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed
assert ( | ||
"sig=" not in line | ||
expected_token_prefix |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My bad - I meant this URL iirc: #2458 (comment)
Generally speaking we may want to test that everything is masked correctly in the Async version as well - and those tests were not applied to async.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed
@@ -165,3 +168,19 @@ def test_query_large_result_set(conn_cnx, db_parameters, ingest_data): | |||
"Expected three telemetry logs (one per query) " | |||
"for log type {}".format(field.value) | |||
) | |||
|
|||
aws_request_present = False | |||
expected_token_prefix = "X-Amz-Signature=" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My bad - I meant this URL iirc: #2458 (comment)
Generally speaking we may want to test that everything is masked correctly in the Async version as well - and those tests were not applied to async.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed
0a25d12
to
228f0a2
Compare
511d01e
to
24ac817
Compare
b48434d
to
a513b10
Compare
Co-authored-by: github-actions <[email protected]>
a513b10
to
eb23308
Compare
_network.py
Jenkinsfile
aioboto3
andaiobotocore
logs and add tests