Skip to content

Upgrade to snowflake-connector-python 4.0.0 and set cert lock timeout to prevent deadlock #1368

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open

Upgrade to snowflake-connector-python 4.0.0 and set cert lock timeout to prevent deadlock #1368

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
b0ff269
Set client prefetch threads to 1 to prevent deadlock at exit
wiggzz Oct 3, 2025
da152e6
Add changelog entry
wiggzz Oct 3, 2025
696b46e
Lint
wiggzz Oct 3, 2025
f60fb98
Update tests
wiggzz Oct 6, 2025
6047713
Upgrade to snowflake-connector-python 4.0.0 and use ocsp_root_certs_d…
wiggzz Oct 10, 2025
49d318e
Add changleog entry
wiggzz Oct 10, 2025
b380f8f
lint
wiggzz Oct 10, 2025
bc3107f
Merge branch 'main' into wj/snowflake-deadlock-client-prefetch
MichelleArk Oct 24, 2025
e8984c7
Merge branch 'main' into wj/snowflake-deadlock-client-prefetch
colin-rogers-dbt Oct 28, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions dbt-snowflake/.changes/unreleased/Dependencies-20251010-095457.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kind: Dependencies
body: Upgrade snowflake-connector-python to 4.0.0
time: 2025-10-10T09:54:57.268488-05:00
custom:
Author: wiggzz
PR: "1368"
6 changes: 6 additions & 0 deletions dbt-snowflake/.changes/unreleased/Fixes-20251003-140016.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kind: Fixes
body: Set client prefetch threads to 1 to prevent deadlock at exit
time: 2025-10-03T14:00:16.074315-05:00
custom:
Author: wiggzz
Issue: "1368"
4 changes: 2 additions & 2 deletions dbt-snowflake/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@ classifiers = [
dependencies = [
"dbt-common>=1.10,<2.0",
"dbt-adapters>=1.16.6,<2.0",
# lower bound pin due to platform_detection_timeout_seconds parameter
"snowflake-connector-python[secure-local-storage]>=3.17.2,<4.0.0",
# lower bound pin due to CVE-2025-24794
"snowflake-connector-python[secure-local-storage]>=4.0.0,<5.0.0",
"certifi<2025.4.26",
# add dbt-core to ensure backwards compatibility of installation, this is not a functional dependency
"dbt-core>=1.10.0rc0",
Expand Down
1 change: 1 addition & 0 deletions dbt-snowflake/src/dbt/adapters/snowflake/connections.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -410,6 +410,7 @@ def connect():
insecure_mode=creds.insecure_mode,
platform_detection_timeout_seconds=creds.platform_detection_timeout_seconds,
session_parameters=session_parameters,
ocsp_root_certs_dict_lock_timeout=10, # cert lock can cause deadlock without timeout, see https://github.com/snowflakedb/snowflake-connector-python/issues/2213
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a reasoning behind 10? Can we make this a user configurable value?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My surface level understanding is that this is the amount of time that the connector will wait to obtain a lock to interact with some certs it uses during the request. I doubt that this will ever take longer than 10 seconds unless there is a deadlock, in which case we'll fail after 10 seconds. So I doubt there is a reason a user would care or want to configure it, I am personally a little confused why the connector exposes this at all. I think this should have been an implementation detail. But I may be misreading it, I'll try to take a deeper look tomorrow.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, we can go with that for now and do a fast follow to make this configurable

**creds.auth_args(),
)

Expand Down
14 changes: 14 additions & 0 deletions dbt-snowflake/tests/unit/test_snowflake_adapter.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -293,6 +293,7 @@ def test_client_session_keep_alive_false_by_default(self):
platform_detection_timeout_seconds=0.0,
session_parameters={},
reuse_connections=True,
ocsp_root_certs_dict_lock_timeout=10,
),
]
)
Expand Down Expand Up @@ -327,6 +328,7 @@ def test_client_session_keep_alive_true(self):
platform_detection_timeout_seconds=0.0,
session_parameters={},
reuse_connections=None,
ocsp_root_certs_dict_lock_timeout=10,
)
]
)
Expand Down Expand Up @@ -355,6 +357,7 @@ def test_client_has_query_tag(self):
insecure_mode=False,
platform_detection_timeout_seconds=0.0,
session_parameters={"QUERY_TAG": "test_query_tag"},
ocsp_root_certs_dict_lock_timeout=10,
)
]
)
Expand Down Expand Up @@ -391,6 +394,7 @@ def test_user_pass_authentication(self):
platform_detection_timeout_seconds=0.0,
session_parameters={},
reuse_connections=True,
ocsp_root_certs_dict_lock_timeout=10,
)
]
)
Expand Down Expand Up @@ -426,6 +430,7 @@ def test_authenticator_user_pass_authentication(self):
platform_detection_timeout_seconds=0.0,
session_parameters={},
reuse_connections=True,
ocsp_root_certs_dict_lock_timeout=10,
)
]
)
Expand Down Expand Up @@ -457,6 +462,7 @@ def test_authenticator_externalbrowser_authentication(self):
platform_detection_timeout_seconds=0.0,
session_parameters={},
reuse_connections=True,
ocsp_root_certs_dict_lock_timeout=10,
)
]
)
Expand Down Expand Up @@ -492,6 +498,7 @@ def test_authenticator_oauth_authentication(self):
platform_detection_timeout_seconds=0.0,
session_parameters={},
reuse_connections=True,
ocsp_root_certs_dict_lock_timeout=10,
)
]
)
Expand Down Expand Up @@ -527,6 +534,7 @@ def test_authenticator_private_key_authentication(self, mock_get_private_key):
platform_detection_timeout_seconds=0.0,
session_parameters={},
reuse_connections=True,
ocsp_root_certs_dict_lock_timeout=10,
)
]
)
Expand Down Expand Up @@ -562,6 +570,7 @@ def test_authenticator_private_key_authentication_no_passphrase(self, mock_get_p
platform_detection_timeout_seconds=0.0,
session_parameters={},
reuse_connections=True,
ocsp_root_certs_dict_lock_timeout=10,
)
]
)
Expand Down Expand Up @@ -595,6 +604,7 @@ def test_authenticator_jwt_authentication(self):
platform_detection_timeout_seconds=0.0,
session_parameters={},
reuse_connections=True,
ocsp_root_certs_dict_lock_timeout=10,
)
]
)
Expand Down Expand Up @@ -626,6 +636,7 @@ def test_query_tag(self):
platform_detection_timeout_seconds=0.0,
session_parameters={"QUERY_TAG": "test_query_tag"},
reuse_connections=True,
ocsp_root_certs_dict_lock_timeout=10,
)
]
)
Expand Down Expand Up @@ -656,6 +667,7 @@ def test_reuse_connections_with_keep_alive(self):
platform_detection_timeout_seconds=0.0,
session_parameters={},
reuse_connections=True,
ocsp_root_certs_dict_lock_timeout=10,
)
]
)
Expand Down Expand Up @@ -691,6 +703,7 @@ def test_authenticator_private_key_string_authentication(self, mock_get_private_
platform_detection_timeout_seconds=0.0,
session_parameters={},
reuse_connections=True,
ocsp_root_certs_dict_lock_timeout=10,
)
]
)
Expand Down Expand Up @@ -728,6 +741,7 @@ def test_authenticator_private_key_string_authentication_no_passphrase(
platform_detection_timeout_seconds=0.0,
session_parameters={},
reuse_connections=True,
ocsp_root_certs_dict_lock_timeout=10,
)
]
)
Expand Down