Skip to content

Commit 01b2c17

Browse files
authored
SNOW-1468259 Add connection timeout and read timeout to storage client request (#1988)
* Add connection timeout and read timeout to storage client request * Add changelog entry
1 parent 3013fe8 commit 01b2c17

File tree

3 files changed

+14
-1
lines changed

3 files changed

+14
-1
lines changed

DESCRIPTION.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ Source code is also available at: https://github.com/snowflakedb/snowflake-conne
99
# Release Notes
1010

1111
- v3.12.0(TBD)
12+
- Set default connection timeout of 10 seconds and socket read timeout of 10 minutes for HTTP calls in file transfer.
1213
- Optimized `to_pandas()` performance by fully parallel downloading logic.
1314
- Fixed a bug that specifying client_session_keep_alive_heartbeat_frequency in snowflake-sqlalchemy could crash the connector
1415

src/snowflake/connector/constants.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -399,6 +399,7 @@ class IterUnit(Enum):
399399
TABLE_UNIT = "table"
400400

401401

402+
# File Transfer
402403
# Amazon S3 multipart upload limits
403404
# https://docs.aws.amazon.com/AmazonS3/latest/userguide/qfacts.html
404405
S3_DEFAULT_CHUNK_SIZE = 8 * 1024**2
@@ -410,6 +411,10 @@ class IterUnit(Enum):
410411
S3_CHUNK_SIZE = 8388608 # boto3 default
411412
AZURE_CHUNK_SIZE = 4 * megabyte
412413

414+
# https://requests.readthedocs.io/en/latest/user/advanced/#timeouts
415+
REQUEST_CONNECTION_TIMEOUT = 10
416+
REQUEST_READ_TIMEOUT = 600
417+
413418
DAY_IN_SECONDS = 60 * 60 * 24
414419

415420
# TODO: all env variables definitions should be here

src/snowflake/connector/storage_client.py

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,13 @@
1919

2020
import OpenSSL
2121

22-
from .constants import HTTP_HEADER_CONTENT_ENCODING, FileHeader, ResultStatus
22+
from .constants import (
23+
HTTP_HEADER_CONTENT_ENCODING,
24+
REQUEST_CONNECTION_TIMEOUT,
25+
REQUEST_READ_TIMEOUT,
26+
FileHeader,
27+
ResultStatus,
28+
)
2329
from .encryption_util import EncryptionMetadata, SnowflakeEncryptionUtil
2430
from .errors import RequestExceedMaxRetryError
2531
from .file_util import SnowflakeFileUtil
@@ -280,6 +286,7 @@ def _send_request_with_retry(
280286
while self.retry_count[retry_id] < self.max_retry:
281287
cur_timestamp = self.credentials.timestamp
282288
url, rest_kwargs = get_request_args()
289+
rest_kwargs["timeout"] = (REQUEST_CONNECTION_TIMEOUT, REQUEST_READ_TIMEOUT)
283290
try:
284291
if conn:
285292
with conn._rest._use_requests_session(url) as session:

0 commit comments

Comments
 (0)